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INTRODUCTION 


Cigarette  smoking  remains  the  leading  cause  of  preventable  disease  and  death  in  the 
United  States,  contributing  to  the  deaths  of  more  than  443,000  people  each  year1.  It  is  also 
responsible  for  80%  of  lung  cancers,  and  30%  of  all  cancers  nationwide.  The  U.S.  Preventive 
Services  Task  Force  has  labeled  tobacco  cessation  as  one  of  the  highest  priority  services  with  the 
lowest  delivery  rate".  Smoking  and  associated  nicotine  dependence  represents  a  chronic 
condition  that  can  be  very  difficult  to  successfully  treat.  An  estimated  53. 1%  of  smokers  report 
that  they  have  stopped  smoking  for  at  least  24  hours  in  the  previous  year  ;  however,  actual  rates 
of  long-term  cessation  are  substantially  lower4. 

One  segment  of  the  U.S.  population  with  a  disproportionately  high  smoking  rate  is  the 
military.  A  survey  of  U.S.  troops  deployed  to  Iraq  and  Afghanistan,  found  that  nearly  40% 
smoked  at  least  one-half  pack  of  cigarettes  per  day  —  with  nearly  half  of  smokers  reporting  that 
they  started  or  resumed  smoking  during  their  deployment5.  In  addition,  more  than  12%  of  all 
military  personnel  (14.5%  among  males)  report  using  smokeless  tobacco6.  Many  Veterans  begin 
smoking  while  enrolled  in  the  military,  so  it  is  not  surprising  that  the  smoking  rate  among 
Veterans  (34%)  also  exceeds  that  of  the  general  population7.  Previous  studies  report  an  historic 
smoking  rate  of  74%  among  Veterans,  compared  with  48%  among  non- Veterans  .  Smoking 
contributes  to  high  morbidity  and  mortality  rates  among  Veterans9,  as  well  as  nearly  $1  billion 
annually  of  lost  productivity  in  the  military10.  In  his  1997  economic  analysis,  Harris  concluded 
that  if  all  Veterans  who  ever  smoked  filed  disability  claims — even  if  they  started  regular 
cigarette  use  prior  to  military  service —  the  total  potential  costs  due  to  all  smoking-related 
diseases  would  equal  $18.4  to  $19.8  billion11. 
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High  rates  of  smoking  among  military  personnel  persist  after  discharge  from  military 
service10.  Overall,  smoking  cessation  interventions  for  military  personnel  have  resulted  in 
abstinence  rates  between  12%  and  19%,  which  is  lower  than  those  for  civilian  populations  ’  . 
Both  the  DoD  and  the  Department  of  Veterans  Affairs  (VA)  have  made  smoking  cessation  a 
priority  for  health  promotion  and  disease  prevention.  In  1993,  the  Air  Force  set  a  smoke-free 
goal  for  1998  and  in  1992  the  Navy  set  one  for  2000.  These  goals  have  not  yet  been  achieved14. 
To  date,  only  Veterans  ages  75  or  older  have  met  the  Year  2000  goal  for  fewer  than  15%  tobacco 
users15. 

This  project  aimed  to  improve  the  health  of  Veterans  through  the  reduction  of  smoking 
behaviors  by  utilizing  a  theory-based,  computerized,  tailored  intervention  (CTI)  with  text 
messaging  feedback.  It  was  theorized  that  the  ubiquity  and  sophistication  of  today’s  wireless 
mobile  technologies  would  represent  new  modes  of  delivery  for  empirically  based  smoking 
cessation  and  other  behavioral  health  interventions.  A  two-group  3-month  randomized  controlled 
pilot  study  was  conducted  to  determine  the  effectiveness  of  adding  tailored  text  messages  to  a 
web-based  CTI  to  decrease  smoking  in  Veterans.  The  following  hypotheses  were  examined: 
Hypothesis  1:  The  structure  and  content  of  the  cell  phone-based  CTI  for  smoking 
cessation  will  be  appropriate  and  relevant  to  Veterans. 

Hypothesis  2 :  Cell  phone-based  CTI  will  facilitate  greater  smoking  cessation  behavior 
change  than  assessment  only  on  a  web-based  CTI. 

A  list  of  Study  Personnel  and  a  compilation  of  Milestones  are  attached  as  Appendices  A  and 
B.  The  study  Timeline  is  attached  as  a  Gantt  Chart  in  Appendix  C. 
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METHODS 


This  randomized,  prospective  pilot  employed  a  2  (group)  x  3  (timepoint)  research  design. 
Veteran  smokers  were  recruited  nationally  from  direct  mailings,  Facebook  ads,  community 
newsletters,  posters,  and  flyers.  In  addition  to  the  randomized  control  trial  (RCT),  a  focus  group 
(n=9)  and  individual  usability  tests  (n=4)  were  conducted  in  order  to  assess  the  acceptability  of  the 
graphic  user  interface,  questions,  tailoring  and  program  content  to  Veteran  smokers. 

While  behavioral  health  interventions  have  traditionally  been  carried  out  without  the  use  of 
technology,  steady  advances  in  behavioral  science,  communications  and  computer  technology  have 
contributed  to  the  development  of  individualized  CTIs  that  motivate  behavioral  change16. 

Computer-  and  Internet-based  interventions  have  performed  well  in  several  efficacy  studies  .  In 
fact,  randomized  controlled  trials  have  supported  the  efficacy  of  computer  and  Internet-based 
interventions  as  comprehensive  interventions,  i.e.,  with  minimal  or  no  clinician  contact,  for  panic 
disorder  ,  depression  "  ,  weight  loss  ,  and  diabetes  self-management  . 

Computerized  interventions  have  several  potential  advantages  over  non-computerized 
protocols  .  First,  computer-based  interventions  are  often  designed  to  interact  directly  with  users 
and  can  be  tailored  to  the  needs  of  a  diverse  group  of  participants.  This  allows  for  personalization  of 
recommendations  with  minimal  burden  of  superfluous  material.  Second,  precise  user  data  (e.g.,  time 
burden  on  users,  answers  to  knowledge  questions)  are  more  readily  collected  via  interactive 
computerized  interventions  relative  to  other,  non-computerized  “self-help”  methods,  such  as 
bibliotherapy  or  videotape  protocols.  Third,  Internet-based  interventions  can  reach  a  large 
population  at  relatively  low  cost  and  can  be  accessed  wherever  computers  are  available,  whereas 
traditional  self-help  interventions  may  be  less  accessible  for  portions  of  the  population.  Fourth,  they 
can  be  accessed  privately  from  individuals’  homes  and  completed  at  users’  own  pace.  Finally,  they 
can  be  easily  adapted  as  empirical  findings  are  updated.  Moreover,  adaptive  interventions  that  offer 
multiple  contacts  in  which  messages  are  dynamically  tailored  are  more  effective  than  traditional 
one-size-fits  all  interventions.  Further,  computer  and  Internet-based  behavioral  health  yield  equally 
effective  treatment  outcomes  compared  to  self-help  interventions  delivered  via  other  methods"  . 

Advanced  CTIs,  such  as  the  one  used  in  this  study,  have  been  previously  developed  for 
civilian  adults  with  NIH  funding  by  our  research  collaborators,  Pro-Change  Behavior  Systems. 

These  systems  employ  empirical  databases  consisting  of  data  collected  from  thousands  of 
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participants  and  heuristics  provide  the  surface  knowledge  used  to  establish  decision  rules  that  guide 
the  development  of  individualized  interventions  tailored  on  behavior  change  theory  variable  ’  . 

Additionally,  the  low-cost  CTI  has  the  potential  to  facilitate  active  engagement  in  the  change 
process  and  significantly  enhance  the  benefits  of  traditional  treatment. 

While  CTIs  have  been  utilized  for  some  time,  delivering  such  programs  through  the 
Internet  is  still  in  an  early  stage  of  development  .  CTIs  have  evolved  from  distributing  tailored 
feedback  via  printed  reports,  to  CD-ROM  based  multi-media  programs,  to  computer  kiosks,  and 
now  via  the  web.  Web-based  CTIs  are  particularly  beneficial  for  intervening  with  some  mental 
health  issues  because  they  offer  anonymity  ’  ,  reduce  fear  of  stigma,  and  increase  self- 

3 1  32 

disclosure  ’  .  CTIs  can  be  more  engaging,  allowing  participants  to  control  their  learning 

_ 'J  c 

environment,  move  at  their  own  pace,  and  allow  access  to  sensitive  information'  .  They  can 
also  potentially  increase  retention  rates  by  increasing  convenience  and  allowing  doses  of 

q i /: 

interventions  as  needed  .  Additionally,  advanced  CTIs  employ  empirical  databases  consisting  of 
data  collected  from  thousands  of  participants  and  heuristics.  These  databases  provide  the  basis 
for  decision  rules  that  guide  the  development  of  individualized  interventions  tailored  according 
to  behavior-change  theory  variables  ’  . 

The  web  is  identified  as  being  the  most  powerful  channel  for  delivering  feedback, 
offering  the  most  access  to  expert  guidance,  and  holding  promise  for  boosting  efficacy  of  CTIs. 
Lustria  reviewed  key  components  of  30  existing  CTIs  delivered  over  the  web.  They  found  that 
there  is  a  great  range  in  what  CTIs  include,  from  simple  screening  tools  to  more  complex  theory 
based  tailoring  with  iterative  assessments.  The  CTI  used  in  this  study  is  composed  of  the 

TO 

strongest  key  components  found  by  Lustria  and  colleagues  ,  including  theoretically  guided 
tailoring,  dynamic  tailoring  across  multiple  contacts,  and  self-directed  delivery. 

The  theoretical  framework  upon  which  the  CTI  system  is  based  is  the  Transtheoretical 
Model  of  Behavior  Change  (TTM)  —  one  of  the  leading  behavior  change  theories  .  The  TTM 
is  a  comprehensive  model  of  behavior  change  that  integrates  diverse  psychological  constructs 
(i.e.,  stage  of  change,  decisional  balance,  process  of  change,  and  self-efficacy)  to  explain  and 
predict  how  and  when  individuals  change  their  health  behaviors.  Several  clinical  trials  have 
documented  the  ability  of  TTM  interventions  to  recruit,  retain,  and  effect  change  across  a 
number  of  health  behaviors,  including  smoking40'41,  stress42,  depression  prevention43,  exercise44, 
diet45,  sun  exposure46,  alcohol47,  weight  management48,  and  multiple  health  behaviors49"51. 
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Clinical  research  that  has  applied  the  TTM  to  Vietnam  veterans  with  PTSD  has  also  shown 
impacts  on  readiness  to  change,  perceived  treatment  relevance,  attendance  at  group  treatment 
sessions,  and  attrition  . 

TTM-based  CTI’s  have  also  generated  much  higher  rates  of  participation  (e.g.,  65%  to 
85%)  for  problems  like  smoking,  stress,  and  obesity  than  the  2%  to  20%  rates  commonly  found 
with  action-oriented  clinic-based  treatments  (e.g.,40,42,48-50).  Further,  those  who  traditionally  are 
at  greatest  risk  for  dropping  out,  participants  in  the  precontemplation  (PC;  not  ready  to  change  in 
the  next  six  months)  stage  completed  CTIs  at  the  same  high  rate  as  those  prepared  to  take  action. 
In  mandated  group  therapy  for  partner  abuse,  randomly  adding  three  sessions  of  CTI’s  nearly 
doubled  the  percentages  of  perpetrators  who  voluntarily  sought  appropriate  help  like  couples 
therapy  (50%  vs.  24%)  or  group  therapy  (36%  vs.  18%)53. 

CTIs  are  a  best  practice  for  behavior  change  science54  as  they  are  highly  individualized  to 
the  participant’s  needs,  yet  maintain  a  standardized  quality55.  Tailored  communications  have 
been  shown  to  be  more  engaging,  effective  in  building  self-efficacy,  and  improving  health- 
behaviors  than  non-tailored  communications  ,  and  useful  in  producing  more  acceptable  and 
effective  interventions56'57.  In  a  2010  meta-analysis,  Krebs  et  al.58  found  that  across  88  CTIs  for 
health  behavior  change  in  the  areas  of  smoking,  exercise,  healthy  eating,  and  mammography 
screening,  a  significant  effect  size  was  found  (g=0.17)  —  a  small  to  medium  effect  for 
population  based  interventions.  As  outlined  by  Hester  et  al.59,  CTIs  require  little  or  no  clinician 
involvement,  which  can  increase  feasibility  and  cost-effectiveness;  they  can  provide  personalized 
feedback  in  a  timely  and  visually  engaging  manner;  they  can  deliver  feedback  more  consistently, 
based  on  assessment  data  and  decision  rules;  they  can  store  the  data  to  chart  changes  over  time; 
and  they  can  be  widely  disseminated  while  preserving  fidelity.  CTIs  combine  the  advantages  of  a 
clinic  approach  (individualized  interactions  with  the  goals  of  a  public  health  approach  (targeting 
large  population  segments). 

A.  Procedures 

All  procedures  were  approved  by  the  VA  Pacific  Islands  Health  Care  System 
(VAPIHCS),  the  U.S.  Anny  Medical  Research  and  Materiel  Command’s  Human  Research 
Protection  Office  (HRPO). 
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A.l  Focus  Group 

No  TTM-based  CTIs  have  been  developed  specifically  for  or  tailored  to  Veterans.  The 
focus  group  for  this  study,  part  of  a  larger  project60,  was  intended  to  bridge  this  gap  by 
evaluating  the  applicability  of  an  evidence-based  CTI  to  address  smoking  in  Veterans,  the  results 
of  which  may  be  used  for  future  adaptation. 

Participants  were  recruited  from  Veteran  communities  in  Hawaifi,  and  were  screened  for 
eligibility  based  on  criteria  outlined  in  Table  1.  Nine  Veterans  participated  in  the  smoking 
cessation  focus  group  to  evaluate  the  CTI  smoking  cessation  program. 

All  procedures  were  approved  by  the  two  institutional  review  boards  involved.  The  focus 
group  methodology  was  infonned  by61.  Two  practice  sessions  were  conducted  to  familiarize 
research  staff  with  the  procedures  and  to  finalize  the  protocol.  Each  focus  group  began  with  the 
informed  consent  process  and  completion  of  a  brief  assessment,  followed  by  introductory 
questions  and  the  evaluation  of  the  CTI  program  using  a  discussion  guide  developed  for  the 
smoking  cessation  program.  The  focus  group  was  led  by  a  trained  facilitator,  with  an  assistant 
facilitator  and  two  note -takers  help  with  the  process.  The  focus  group  was  recorded  using  an 
audio  device,  and  lasted  approximately  two  hours.  A  $25  gift  card  was  provided  to  each 
participant  at  the  end  of  the  meeting. 

A.2.  Usability  Testing 

After  the  CTI  program  had  been  adapted  and  beta-tested,  its  acceptability  and  usability 
was  examined  to  provide  a  scientific  assessment  of  user  errors,  misunderstandings  of  content, 
navigation  problems,  and  subjective  satisfaction  . 

Three  separate  usability  tests  were  conducted  for  the  CTI  smoking  cessation  program 
using  a  combination  of  the  Think  Aloud  protocol63  and  Wizard  of  Oz  approach64’6^.  Briefly, 
participants  were  asked  for  initial  reactions  to  the  introduction  to  the  CTI  system  and  program. 
Participants  were  then  asked  to  logon  to  the  CTI  program,  continue  through  the  baseline 
assessment,  and  narrate  their  behavior  as  they  progressed  through  the  program.  Usability  testing 
software  (Morae®)  was  used  to  take  notes  and  record  participant  interactions  with  the  program, 
including  paths  taken,  reactions  to  content  and  videos,  instances  of  confusion  about  how  to 
navigate  through  the  web-based  multimedia  program  menu,  how  long  users  spent  on  each 
exercise,  whether  assessments  are  successfully  completed,  sources  of  frustration,  what  aspects  of 
the  program  seemed  to  be  missing,  and  overall  level  of  satisfaction.  Participants  provided 
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qualitative  and  quantitative  feedback  on  overall  presentation  and  usability,  as  well  as  quality  of 
the  program,  navigation,  ease  of  use,  attractiveness,  etc.  All  interview  data  were  coded, 
categorized,  and  summarized  in  a  feedback  report  so  that  revisions  to  the  system  could  be 
adapted. 

Two  additional  usability  interviews  were  conducted  using  the  same  procedures  described 
above  to  ensure  that  the  revisions  made  based  on  the  first  round  of  usability  testing  were 
adequate.  All  usability  interview  participants  received  a  $50  gift  card  at  the  end  of  their 
particular  session  as  reimbursement  for  their  time  and  contribution  to  the  study. 

A. 3.  Randomized  Control  Pilot  Trial 

After  registration  with  the  system,  participants  were  asked  to  provide  their  10-digit  cell¬ 
phone  numbers.  Participants  who  verified  their  cell  phone  number  by  sending  an  automated 
response  to  the  system  were  considered  enrolled.  At  all  three  timepoints,  participants  completed 
a  brief  Health  Risk  Intervention  (HRI),  two  additional  smoking  assessments,  and  the  smoking 
cessation  (SC)  intervention.  The  HRI  assessment  determined  baseline  readiness  to  change  for 
smoking  and  several  other  related  health  behaviors,  and  the  smoking  cessation  intervention 
provided  assessment  and  feedback  about  participants’  confidence  to  quit  smoking,  decisional 
balance,  and  use  of  various  change  strategies. 

Following  the  baseline  HRI  and  SC  intervention,  the  CTI  system  randomized  participants 
into  either  the  treatment  group  (Tx),  who  also  received  the  CTI  only,  or  the  treatment-plus  group 
(Tx-plus),  who  received  the  CTI  and  tailored  text  messages.  The  study  was  conducted  as  entirely 
anonymous.  All  participants  were  given  three  months’  access  to  the  web-based  CTI  system.  The 
Tx  participants  received  only  the  online  feedback  narratives/report  that  all  participants  received 
upon  completion  of  the  online  Smoking  Cessation  intervention.  Tx-plus  participants  also 
received  individualized  text  messages  on  their  cell  phones  2-5  times  per  week  (the  number  of 
messages  correlated  with  the  participant’s  stage  of  change)  that  provide  expert  guidance, 
encouragement,  and  reminders  that  support  their  smoking  cessation  efforts.  These  messages 
were  developed  using  the  empirically  derived  decision  rules  of  the  web-based  smoking  cessation 
CTI  that  were  established  by  comparing  each  participant’s  responses  on  the  baseline  assessment 
to  a  large  comparative  sample  of  other  individuals  in  that  stage  (nonnative  comparisons).  The 
CTI  system  can  generate  over  150  unique  feedback  nanatives  at  baseline  and  more  than  20,000 
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unique  narratives  after  follow-up  assessments.  This  capability  ensures  that  participants  do  not 
receive  the  same  feedback  each  time  they  receive  a  text  message  from  the  system. 

All  text  message  feedback  was  stage -tailored,  and  offered  strategies  for  facilitating  the 
change  process  relevant  for  the  target  behavior.  For  example,  a  participant  in  the  Contemplation 
stage  (C)  with  below  average  confidence  would  receive  tailored  feedback  on  their  level  of 
confidence  as  well  as  strategies  for  boosting  their  confidence  such  as  setting  small  goals  and 
asking  a  friend  to  encourage  them  not  to  use  tobacco.  An  example  of  an  encouraging  text 
message  may  be  “Do  something  nice  for  yourself  when  you  take  small  steps  toward  quitting,  like 
cutting  back  on  the  amount  you  smoke.  Ask  a  friend  to  help  you.” 

The  follow-up  one -month  assessment  assessed  whether  participants  had  progressed  or 
relapsed  in  their  stages  of  change,  confidence  to  quit  using  tobacco,  decisional  balances,  or  use 
of  strategies  for  change.  In  addition  to  normative  feedback,  ipsative  feedback  was  provided 
(compared  to  self)  that  reinforced  progress  the  participant  had  made  on  the  TTM  constructs  since 
the  baseline  smoking  cessation  assessment,  and  explained  what  specific  steps  the  participant 
could  take  to  progress  further.  For  example,  a  participant  who  progressed  to  the  Action  stage  (A) 
who  was  practicing  an  appropriate  amount  of  reinforcement  management  for  smoking  cessation 
would  be  advised  to  keep  up  the  good  work,  and  consider  what  reinforcements  would  help  him 
or  her  to  maintain  their  non-smoking  status.  Suggestions,  such  as  realizing  the  health  and 
physical  benefits,  and  including  family  and  friends  in  their  healthy  lifestyle,  were  offered.  Tx- 
plus  participants  then  received  stage-matched  text  message  feedback  on  their  cell  phones  based 
on  the  one-month  assessment  results. 

For  their  participation  in  the  study,  all  participants  received  up  to  $65  in  gift  cards  as 
reimbursement  for  their  time  and  effort.  They  received  a  $20  gift  card  after  completing  the 
baseline  assessment,  and  another  $20  gift  card  after  completing  the  1 -month  assessment 
measures.  Those  who  completed  post-study  assessment  measures  at  the  conclusion  of  the  active 
phase  of  the  study  (3  months)  received  a  $25  gift  card.  In  addition,  Tx-plus  participants  received 
an  additional  $10  in  gift  cards  ($5  gift  card  per  month  up  to  2  months)  to  support  their  text 
messaging  service  fees  while  they  are  active  participants  in  the  research  project. 
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B.  Measures 

B.  1 .  Demographics.  This  self-report  questionnaire  was  developed  for  the  project  to  obtain 
participant  demographic  information,  including  race/ethnicity,  age,  gender,  education,  marital 
status,  rank  when  retired,  combat  theater(s)  served,  total  number  of  months  in  theater(s), 
smoking  history,  etc.  Participants  were  also  asked  to  note  any  other  treatments  or  smoking 
cessation  programs  that  they  have  used  in  the  past. 

B.2.  Health  Risk  Intervention  Survey  (HRI).  This  survey  consisted  of  questions  about  health 
behaviors  related  to  smoking  cessation.  The  behavior  infonnation  was  used  to  assess  the 
participant’s  stage  of  change  relative  to  smoking  cessation  and  other  smoking-related  health 
behaviors.  This  included  stage  of  change  assessments  for  exercise,  healthy  eating,  alcohol 
misuse,  stress  management,  and  depression  prevention.  Participants  completed  the  HRI  at  each 
timepoint. 

B.3.  Fagerstrom  Nicotine  Dependence  Scale  (FTND).  The  FTND66  is  the  most  widely  used 
tool  for  assessing  severity  of  nicotine  tolerance  and  dependence.  It  is  a  six-item,  self-report  scale 
assessing  severity  of  nicotine  tolerance  and  dependence  in  smokers.  The  FTND  is  modestly 
correlated  with  biochemical  indicators  of  exposure  to  cigarette  smoke,  including  exhaled  air 
carbon  monoxide  and  plasma  cotinine.  The  FTND  is  correlated  with  duration  of  smoking 
(r=.36),  smoking  intensity  (r=. 45),  and  saliva  cotinine  (r=.40).  Internal  consistency  for  this  scale 
was  an  average  of  a=.95  for  the  three  timepoints.  Participants  completed  the  FTND  at  each 
timepoint. 

B.4.  Questionnaire  on  Smoking  Urges  (QSU).  The  QSU  was  used  to  measure  subjective 
changes  in  positively  and  negatively  reinforced  craving  levels  in  response  to  smoking  stimuli  . 

A  brief  10-item  version  of  the  QSU  was  used  in  this  study.  Internal  consistency  for  this  scale 
was  an  average  of  a=.62  for  the  three  timepoints.  Participants  completed  the  QSU- 10  at  each 
timepoint. 

B.5.  TTM  Smoking  Cessation  Assessment.  This  online  self-report  measure  that  assessed  the 
four  TTM  core  constructs,  i.e.,  Stage  of  Change,  Decisional  Balance,  Processes  of  Change,  and 
Self  Efficacy,  in  relation  to  smoking  cessation.  Stage  of  Change  (2  items)  assessed  readiness  to 
quit  smoking68,  e.g.,  “No,  I  intend  to  quit  in  the  next  6  months.”  Decisional  Balance 69  measured 
the  relative  importance  of  the  advantages  (pros)  and  disadvantages  (cons)  in  an  individual’s 
decision  to  quit  smoking.  Using  a  5-point  Likert  response  scale  (l=not  important;  5=extremely 
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important),  participants  rated  6  items  on  their  relative  importance,  e.g.,  “Quitting  can  improve 

70 

my  appearance.”  Internal  consistency  was  a=.83  for  both  pros  and  cons.  Processes  of  Change 
consisted  of  445  items  that  assessed  10  cognitive,  affective,  experiential,  and  behavioral 
techniques  used  by  individuals  to  facilitate  the  change  process  for  smoking  cessation,  e.g.,  “You 
made  a  commitment  to  quit.”  Responses  were  given  on  a  5-point  Likert  scale  of  frequency 
(l=never;  5=repeatedly).  .”  Internal  consistency  for  the  10  processes  of  change  scales  ranged 
from  a=.61  to  .91.  The  final  component,  Self-Efficacy,  (8  items)71  assessed  an  individual’s 
confidence  level  to  refrain  from  smoking  even  in  difficult  situations,  e.g.,  “When  you  have  just 
finished  a  meal.”  Confidence  level  ratings  are  made  on  a  5-point  Likert  scale  (1=  not  at  all; 
5=extremely  confident).  Internal  consistency  for  self-efficacy  was  a=.89.  Participants  completed 
the  TTM  assessment  measures  at  each  timepoint. 

B.6.  User  Satisfaction.  This  16-item  questionnaire  was  used  to  obtain  user  feedback  about  the 
CTI  intervention  materials  and  is  typically  used  by  Pro-Change  during  the  program  development 
process.  Twelve  items  were  rated  on  a  5 -point  Likert  scale  from  1  ( Strongly  Disagree )  to  5 
(, Strongly  Agree )  and  assessed  multiple  dimensions  of  the  intervention  materials  including  ease 
of  use,  clarity  of  items  and  feedback,  attractiveness,  appropriateness  of  tailoring,  degree  of 
interest  and  enjoyment,  ability  to  convey  information,  ability  to  change  attitudes,  helpfulness, 
ability  to  elicit  appropriate  action,  and  credibility.  Total  possible  scores  range  from  12  to  60, 
with  higher  scores  indicating  higher  user  satisfaction  with  the  intervention  materials.  The  last 
four  questionnaire  items  are  open-ended  questions  requesting  participants  to  provide  suggestions 
and  criticisms  of  the  intervention  materials.  Study  participants  this  questionnaire  at  the  end  of 
the  study. 
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C.  Data  Analysis  Plan 

C.l.  Focus  Group  Qualitative  Analysis  (Hypothesis  One) 

The  focus  group  analysis  followed  the  guidelines  recommended  by  Krueger64</sup>. 
Immediately  after  each  focus  group,  the  facilitator,  assistant  facilitator,  and  note -takers  debriefed 
the  process  and  noted  group  dynamics.  The  audio  recording  was  transcribed  and  all  identifiers 
were  removed  to  protect  confidentiality.  The  transcript  created  was  then  compared  with  the  notes 
to  ensure  completeness  and  accuracy.  The  analysis  entailed  breaking  the  transcript  into  meaning 
units  and  then  grouping  similar  meaning  units  to  form  themes.  To  be  considered  a  theme,  an  idea 
had  to  be  mentioned  at  least  twice.  After  extraction  of  themes,  major  themes  were  identified  and 
examined  in  terms  of  the  applicability  of  existing  CTI  programs  to  Veterans.  Particularly, 
themes  specific  to  Veterans  were  summarized  to  direct  future  adaptation. 

C.l  Descriptives  (Hypothesis  Two) 

All  statistical  analyses  were  conducted  using  IBM®  SPSS®  Version  20.0.  Demographics 
were  analyzed  using  descriptive  statistics,  frequencies,  crosstabs,  and  t-tests.  The  significance  level 
for  all  tests  was  set  at  two-tailed  a=0.05. 

C.3  Outcomes  Analysis  (Hypothesis  Two) 

All  statistical  analyses  were  conducted  using  IBM®  SPSS®  Version  20.0.  Multivariate 
Analysis  of  Variance  (MANOVA)  was  used  as  the  omnibus  test  for  all  continuous  dependent 
variables,  with  stage  of  change  and  group  assignment  acting  as  the  independent  variables. 
Discriminant  Function  Analysis  was  used  to  examine  predictors  of  change  for  both  groups  across 
the  three  timepoints.  Chi-square  was  used  to  examine  discrete  and  categorical  variables,  including 
stage  of  change,  and  other  behavioral  risk  factors. 

C.4  User  Satisfaction  (Hypothesis  One) 

A  review  of  all  quantitative  and  qualitative  feedback  provided  by  participants  in  the  user 
Satisfaction  survey  was  conducted  to  determine  if  the  CTI  was  appropriate  and  relevant  to  Veterans. 
Ratings  will  also  be  analyzed  by  stage  of  change  to  determine  if  positive  feedback  about  the  system 
is  associated  with  progress  over  time  and/or  actual  behavior  change. 
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RESULTS 


A.  Participants  (Focus  Group  Only) 

Ten  Veterans  (all  male)  volunteered  to  participate  in  the  smoking  focus  group.  One 
individual  did  not  attend.  All  participants  were  male.  Table  2  provides  a  demographics  summary. 

B.  Participants  (Usability  only) 

Five  Veterans  (all  male)  volunteered  to  participate  in  the  usability  interviews.  Table  3 
provides  a  demographics  summary. 

C.  Participants  (RCT  only) 

A  total  of  1,174  Veterans  registered  to  learn  more  about  the  study.  Of  those,  268  did  not  consent 
to  join  the  study,  48  remained  inactive,  95  failed  to  complete  the  initial  assessment,  39  were 
removed  for  providing  duplicate  phone  numbers,  55  were  removed  for  suspicious  behavior,  213  did 
not  respond  to  a  verification  message,  and  2  individuals  assigned  to  the  Tx-plus  group  requested 
that  all  text  messages  be  stopped.  Of  the  remaining  individuals,  210  were  randomized  to  the  Tx-plus 
group,  and  236  were  randomized  to  the  Tx  group.  The  following  analyses  will  be  conducted  on  the 
remaining  446  participants  completed  the  baseline  assessment  and  intervention  program. 

C.l  Demographics 

The  sample  (n=446)  was  composed  of  individuals  from  all  but  three  U.S.  states  and 
Puerto  Rico.  There  were  no  verified  participants  from  New  Hampshire,  New  Mexico,  and 
Vennont.  With  the  exception  of  gender  (%  (1)=4.99,  p<.05),  there  were  no  significant  differences 
between  the  composition  of  the  Tx  and  Tx-plus  groups.  Baseline  demographics  are  provided  in 
Table  4. 

C.2  Military  Service 

The  study  participants  represented  all  military  service  branches,  with  the  highest 
representation  from  the  Army  (42.8%).  There  were  no  significant  differences  between  the 
composition  of  the  Tx  and  Tx-plus  groups  with  regard  to  military  service 
(see  Table  5). 
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C. 3.  Smoking  History 

Almost  half  (43.3%%)  of  Veterans  started  smoking  cigarettes  after  joining  the  service, 
and  the  majority  (79. 1%)  smoked  between  %  and  1  Vi  packs  per  day.  The  majority  of  participants 
had  made  between  1-5  previous  quit  attempts  (69.1);  however,  almost  20%  had  attempted  to  quit 
smoking  10  or  more  times.  Average  nicotine  dependence  was  “medium,”  and  average  urge  to 
smoke  was  “moderate.”  Stage  of  change  had  a  proportionately  higher  group  of  individuals  in  the 
Contemplation  stage  than  is  commonly  seen  among  smokers.  Most  stages  of  change  studies  have 
found  the  proportion  of  smokers  to  be  40%  in  Precontemplation,  40%  in  Contemplation,  and 
20%  in  Preparation.  This  sample  appears  to  have  some  intention  to  change,  although  not  in  the 
near  future.  There  were  no  significant  differences  between  the  composition  of  the  Tx  and  Tx- 
plus  groups  with  regard  to  smoking  history  or  scale  scores  (see  Table  6  and  7). 

D.  Focus  Group 

In  general,  participants  thought  the  content  of  the  program  was  appropriate  for  Veterans 
who  were  having  difficulty  quitting  smoking.  They  found  the  concept  of  stages  to  be  helpful  for 
self-evaluating  and  stage-tailored  feedback  encouraging  for  progress.  Participants  especially 
liked  aspects  of  the  system  that  acknowledged  their  autonomy  as  Veterans.  The  individually 
tailored  feedback  on  processes  of  change  was  also  considered  to  help  them  adopt  useful 
strategies  or  maintain  effective  strategies  already  employed.  Many  participants  believed  that  the 
included  goal-setting  would  be  very  useful  in  helping  them  move  forward  through  the  stages.  In 
particular,  small  steps  toward  specific  goals  were  especially  appealing  because  it  increased  the 
manageability  of  behavior  change. 

Scientific  and  user-friendly  language  each  gained  some  support,  with  a  preference  for  a 
combination.  They  particularly  preferred  text  that  is  clear,  concise,  specific,  infonnative,  and 
easy  to  understand.  Most  graphics  in  the  programs  were  considered  relevant  to  the  infonnation 
presented  in  the  corresponding  text  and  helpful  for  users  to  better  understand  the  content  or  feel 
more  positive  about  changing. 

Although  the  majority  of  the  content  was  appropriate,  some  was  considered  difficult  or 
inappropriate.  When  reviewing  the  screenshots,  participants  had  difficulty  understanding  a 
couple  of  professional  tenns  without  a  definition  or  explanation,  such  as  “transtheoretical”  and 
“contemplation”.  They  also  had  difficulty  relating  to  the  function  of  the  “pros  and  cons” 


W81XWH-09-2-0138  (Final  Report)  /  page  17 


exercise.  In  addition,  they  indicated  that  some  benefits  (e.g.,  improvement  in  appearance)  and 
certain  activities  (e.g.,  yoga)  in  the  CTI  may  not  appeal  to  Veterans.  Furthermore,  they  did  not 
like  graphics  that  triggered  combat  memories  or  unhealthy  behaviors,  including  the  beach  at 
sunset  and  cigarettes.  They  also  suggested  that  helping  relationships  were  a  source  of  stress 
rather  than  support  for  Veterans  because  of  difficulty  relating  to  non- Veteran  friends  and  family 
members. 

Some  suggestions  for  adaptation  were  proposed  during  the  discussion.  Inclusion  of  more 
scientific-based  infonnation  in  a  user-friendly  language  was  recommended.  Providing  more 
Veteran-specific  helping  strategies  was  proposed,  such  as  “couples  counseling”  for  post¬ 
deployment  relationship  building  and  avoidance  of  isolation  for  the  depression  prevention. 
Participants  also  suggested  that  the  content  of  the  graphics  be  consistently  related  to  the  written 
information. 

A  copy  of  the  Final  Analysis  Report  is  attached  as  Appendix  D. 

E.  Usability  Interviews 

A  brief  analysis  of  the  usability  video  and  user  feedback  found  that  relevant  usability 
events  fell  into  one  of  the  following  four  categories: 

1 .  HomepageAV elcome  Page  issues 

2.  Errors 

3.  Suggestions 

4.  Positive  Comments 

For  example,  one  usability  issue  with  the  Homepage  was  with  the  lack  of  a  “Next” 
button  on  the  page  for  to  prompt  forward  navigation.  Another  user  wanted  additional 
instructions  on  what  to  do  to  get  from  the  Homepage  to  the  e-Workbook.  Other  comments 
included  revising  the  graphic  images  on  the  home  page  (e.g.,  someone  putting  out  a 
cigarette).  With  the  exception  of  the  addition  of  new  graphic  image,  all  other  suggested 
changes  were  made. 

Errors  that  were  captured  by  the  usability  testing  were  all  user  errors  that  required  the 
user  to  re-read  instructions  in  order  to  move  forward  through  the  system.  For  example,  a 
participant  received  an  error  indicating  that  s/he  was  not  a  returning  user;  the  participant  did 
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not  see  the  “First  Time  Registering”  link.  The  instructions  to  register  and  create  a  new  user 
ID  were  moved  to  a  different  location,  so  that  participants  could  easily  find  them. 

Suggestions  for  improvement  included  a  preference  for  more  photos  throughout  the 
system,  confusion  about  how  to  answer  pros  and  cons  questions,  a  dislike  for  the  color 
scheme,  a  request  to  reduce  the  amount  of  text,  a  recommendation  to  add  drop  down  menus 
to  some  of  the  questionnaires.  Those  suggestions  that  could  be  incorporated  into  the  system 
without  compromising  the  content  or  requiring  a  redesign  of  the  GUI  were  incorporated. 

Finally,  positive  comments  often  reflected  personal  tastes,  such  as  liking  the  color 
scheme,  and  liking  certain  photos.  However,  more  often,  positive  comments  were  associated 
with  the  content  of  the  system,  such  as  the  positive  feedback  and  encouragement,  the 
inclusion  of  the  e-Workbook,  the  printable  feedback  report,  and  the  relevance  of  the  program 
to  Veterans. 

A  copy  of  the  deidentified  user  comments  and  summary  analysis  are  attached  as 
Appendix  E. 

F.  Randomized  Control  Pilot  Study 

Hypothesis  1:  The  structure  and  content  of  the  cell  phone-based  CTI  for 
smoking  cessation  will  be  appropriate  and  relevant  to  Veterans. 

While  data  are  available  only  for  those  participants  who  completed  the  study,  indications 
from  the  User  Satisfaction  Scale,  as  well  as  the  open-ended  questions,  were  that  Veterans  found 
the  CTI  and  tailored  text  messages  to  be  both  appropriate  and  relevant  (see  Table  8).  In  addition, 
participants  in  the  Tx-plus  group  also  indicated  that  the  text  messages  were  helpful,  timely  and 
useful  (see  Table  9). 

Quantitative  analysis  of  ratings  across  the  stages  of  change  found  that  10  of  the  31  items 
had  statistically  significant  mean  differences  (see  Tables  8  and  9).  All  mean  differences  could  be 
ascribed  to  participants  in  the  PC  stage  of  change  at  three  months  rating  the  system  lower  in 
some  areas  than  participants  in  higher  stages.  This  is  not  entirely  unexpected,  given  that  those  in 
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the  PC  stage  of  change  had  no  intention  of  change  their  smoking  behavior,  and  the  items  that 
were  significantly  different  related  to  successful  change. 

Hypothesis  2:  Cell  phone-based  CTI  will  facilitate  greater  smoking  cessation 
behavior  change  than  assessment  only  on  a  web-based  CTI. 

While  446  Veterans  completed  baseline,  participant  retention  at  the  one-month  follow-up 
was  approximately  65%  (n=290).  Individuals  were  sent  text  messages  reminding  them  to  return 
to  the  system  to  complete  the  next  assessment  and  intervention.  Participants  who  had  not 
returned  to  the  program  after  8  reminders  were  no  longer  contacted  and  were  considered 
inactive.  An  additional  55  participants  became  inactive  between  the  one-  and  three-months 
follow-ups.  Although  this  is  only  a  19%  attrition  rate  between  one  and  three  months,  it  represents 
an  overall  retention  rate  of  52.7%. 

Unless  noted,  the  remaining  analyses  will  be  conducted  on  those  participants  for  whom 
there  are  complete  data  (n=235).  This  final  sample  was  evenly  split,  with  1 19  Veterans  in  the  Tx- 
plus  group  (50.6%),  and  1 16  in  the  Tx  group  (49.4%).  A  selection  of  summary  demographics  is 
presented  in  Table  10.  As  previously  seen  with  the  baseline  completion  sample,  there  were  no 
statistically  significant  differences  between  the  two  groups  with  the  exception  of  gender 
composition.  Notably,  when  compared  with  the  baseline  sample,  there  were  two  statistically 
significant  differences  between  the  Tx  groups  those  who  completed  baseline  only  and  those  who 
completed  the  entire  study  through  the  three-month  follow-up.  Those  in  the  Tx  group  who 
completed  the  three-month  follow-up  were  more  likely  to  be  female  (%  (2)=6.67,  p<.05),  and 
either  single  or  living  with  a  partner  (%  (10)=20.35,  jo< .05). 

Smoking  characteristics  and  scale  scores  for  the  sample  of  completers  is  displayed  in 
Table  11.  There  were  no  differences  between  scores  and  stage  distribution  for  the  baseline 
sample  when  compared  with  those  who  completed  the  study. 

F.l.  Tx  Compared  to  Tx-plus  Outcomes 

Several  key  outcome  measures  were  used  to  determine  whether  the  addition  of  text 
messaging  had  a  significant  effect  on  Veterans’  smoking  behavior.  These  measures  include  stage 
of  change,  stage  progression,  increases  in  situational  self-efficacy,  decrease  in  number  of 
cigarettes  per  day,  nicotine  dependence  score,  and  score  on  smoking  urges  scale. 
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Figures  1  to  3  illustrate  the  changes  in  stage  distribution  from  baseline  to  one-month  and 
three-month  follow-ups.  While  the  distributions  are  not  statistically  significantly  different  (due  to 
the  small  sample  size),  the  Tx-plus  group  appears  to  progress  more  quickly  to  action  than  the  Tx 
group. 


Figure  1.  Stage  of  Change  Distribution  at  Baseline  (n=235) 
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Figure  2.  Stage  of  Change  Distribution  at  One  Month  (n=235) 
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Figure  3.  Stage  of  Change  Distribution  at  Three  Months  (n=235) 


In  addition  to  distributions  across  stage  of  change,  the  difference  between  smokers  who 
move  from  risk  (either  PC,  C,  or  PR  stage)  at  baseline  and  those  who  are  no  longer  at  risk  (A  or 
M  stage)  is  also  notable  between  the  two  groups.  At  one  month,  the  proportion  of  Tx-plus 
Veterans  who  were  no  longer  at  risk  was  21.8%,  compared  to  13.8%  who  were  no  longer  at  risk 
in  the  Tx  group.  At  three  months,  43.2%  of  participants  were  no  longer  at  risk,  compared  to 
32.8%  of  the  Tx  group  —  a  difference  of  more  than  10%  (see  Figure  4).  Although  there  were  no 
significant  differences  between  the  two  groups,  individuals  who  quit  smoking  tended  to  be 
younger  than  those  who  remained  in  the  PC  stage  (F(3,100)=10.97,  p  <  .05,  r|  =0.1).  The  mean 
age  for  individuals  who  quit  smoking  was  3 1 .6  (±9.2)  years  in  the  Tx-plus  group  and  3 1 .2  (±6.4) 
in  the  Tx  group.  For  those  who  were  in  the  PC  stage  at  3  months,  the  mean  age  was  40.9  (±13.1) 
years  in  the  Tx-plus  group  and  38.3  (±1 1.4)  in  the  Tx  group. 

Additionally,  from  baseline  to  three  months,  individuals  in  the  Tx  group  had  a  range  of 
stage  movement  from  -2  (regressed  two  stages)  to  2  (progressed  two  stages);  whereas, 
individuals  in  the  Tx-plus  group  had  a  range  of  stage  movement  from  -1  to  3.  This  could  indicate 
that  the  addition  of  text  messages  to  the  intervention  helped  to  reduce  recidivism  and  hasten 
progress  through  the  stages  of  change. 
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Figure  4,  Change  from  Smoking  at  Baseline  to  Not  Smoking  at  3  Months 


Changes  in  smoking  behavior  were  also  evident  as  the  number  of  cigarettes  smoked  per 
day  decreased  across  time.  The  proportion  of  individuals  in  both  groups  who  smoked  more  than 
one  pack  of  cigarettes  per  day  dropped  from  21.3%  at  baseline  to  7.6%  at  three  months.  Changes 
in  situational  self-efficacy,  nicotine  dependence  and  smoking  urges  were  not  significant  between 
groups;  however,  they  were  significant  across  time  for  all  completers  (see  Tables  12  and  13) 
Overall,  QSU  scores  decreased  from  a  mean  of  44.74  (±14.2)  at  baseline  to  a  mean  of 
27.3  (±15.8)  at  three  months  (Wilks’  X,=.55,  p<.05,  r|2=.45).  Nicotine  dependence  scores 
decreased  from  a  mean  of  4.62  (±2.2)  to  3.28  (±2.2)  at  three  months  (Wilks’  X,=.67,  p<.05, 
r|  =.33).  Situational  self-efficacy  increased  from  a  mean  of  20.59  (±7.3)  at  baseline  to  a  mean  of 
28.08  (±8.5)  at  three  months  (Wilks’  X=. 52,  p<.05,  r|2=.48). 

F.2  Utilization  of  TTM  Constructs 

Previous  research  has  determined  that  there  is  differential  use  of  various  TTM  constructs  across 
the  stage  of  change.  For  example  the  pros  of  changing  are  generally  about  1  standard  deviation 
lower  in  the  PC  stage  than  in  the  A  stage;  cons  of  changing  are  generally  about  .5  standard 
deviation  higher  in  PC  than  in  PR;  and  situational  self-efficacy  increases  about  2  standard 
deviations  from  PC  to  M.  In  addition,  there  are  specific  strategies,  or  processes  of  change,  that 
are  utilized  at  various  stages  of  change  and  can  be  associated  with  successful  progression  or 
relapse.  For  example,  experiential  or  cognitive  processes  of  change  [i.e.,  consciousness  raising 
(CR),  dramatic  relief  (DR),  environmental  reevaluation  (ER),  self-liberation  (SL),  and  self¬ 
reevaluation  (SR)]  are  utilized  much  more  in  the  early  stages  of  change  than  are  behavioral  or 
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active  processes  of  change  [i.e.,  counter-conditioning  (CC),  stimulus  control  (SC),  helping 
relationships  (HR),  reinforcement  management  (RM),  and  social  liberation  (SO)].  Utilization  of 
various  constructs  was  compared  across  stages  of  change  and  between  the  two  groups 
(see  Table  14). 

Discriminant  Function  Analysis  was  used  to  determine  which  TTM  constructs  were 
predictive  of  stage  of  change  at  three  months’  follow-up  and  stage  progression  overall  between 
the  two  groups.  The  analysis  included  Pros,  Cons,  DCBL,  and  Self-Efficacy  scores  at  one-month 
follow-up  as  predictor  variables  and  Stage  of  Change  at  three-months  as  the  dependent  variable. 
At  one-month  follow-up,  the  four  predictors  from  correctly  classified  50.4%  of  cases  at  three- 
months’  follow  up  in  the  Tx-plus  group  and  44.0%  of  those  in  the  Tx  group.  These  same 
predictors  at  three  months  correctly  classified  63.9%  of  those  in  the  Tx-plus  group  56.9%  of 
those  in  the  Tx  group. 

Stepwise  linear  multiple  regressions  were  used  to  determine  which  TTM  variables 
accounted  for  the  most  variance  across  stage  of  change  and  number  of  cigarettes  smoked  per  day 
(main  outcomes)  for  each  group.  Both  DCBL  and  self-efficacy  at  one -month  and  three-months 
were  regressed  separately  on  to  the  main  outcomes  variables  at  three-months.  For  both  groups 
only  self-efficacy  at  one  month  was  a  significant  predictor  of  stage  of  change  at  three  months 
(Tx-plus:  F(l,112)=26.89,  p<.05,  adj.  R2=.19;  Tx:  F(l,104)=21.33,  p<.05,  adj.  R2=.16).  Only 
self-efficacy  at  one  month  a  significant  predictor  of  number  of  cigarettes  smoked  at  three  months 
(Tx-plus:  F(l,l  12)=1 9.0,  p<.05,  adj.  R2=.14;  Tx:  F(l,104)=14.5,  p<.05,  adj.  R2=.ll).  Similarly, 
only  self-efficacy  at  three  months  was  a  significant  predictor  of  stage  of  change  across  groups 
(Tx-plus:  F(  1,11 0)=7 1 .4,  p<.05,  adj.  R2=.39;  Tx:  F(l,103)=43.9,  p<.05,  adj.  R2=.29),  and  the 
only  significant  predictor  of  number  of  cigarettes  smoked  at  three  months  (Tx-plus: 

F(l,l  10)=58.2  p<.05,  adj.  R2=.34;  Tx:  F(l,103)=29.17,  p<.05,  adj.  R2=.21). 

Co-Action  Findings 

In  addition  to  smoking,  stage  of  change  was  assessed  at  all  three  timepoints  for  exercise, 
healthy  eating,  stress  management,  depression  prevention,  alcohol  use,  and  sleep  management. 
Although  participants  did  not  receive  an  intervention  on  any  of  these  behaviors,  change  from  risk 
to  no-risk  was  also  evident. 
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Figure  5.  Stage  of  Change  Distribution  for  Exercise  at  Baseline 


More  than  half  of  the  participants  (n=126;  53.6%)  were  in  PC,  C  or  PR  at  baseline  for 
regular  exercise.  Of  those  who  were  at  risk  at  baseline,  53.1%  of  those  in  the  Tx-plus  group  and 
47.5%  of  those  in  the  Tx  group  moved  to  the  Action  (A)  or  Maintenance  (M)  stage. 


Figure  6.  Change  from  Risk  to  No  Risk  From  Baseline  to  Three  Months  for  Exercise 
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Figure  7.  Stage  of  Change  Distribution  for  Healthy  Eating  at  Baseline 
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More  than  70%  of  the  participants  (n=167;  71.1%)  were  in  PC,  C  or  PR  at  baseline  for 
healthy  eating  (i.e.,  eating  a  low-fat  diet  and  proper  caloric  intake).  Of  those  who  were  at  risk  at 
baseline, 

Figure  8.  Change  from  Risk  to  No  Risk  From  Baseline  to  Three  Months  for  Healthy  Eating 
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Figure  9.  Stage  of  Change  Distribution  for  Stress  Management  at  Baseline 


Forty  percent  of  participants  (n=94)  were  in  PC,  C  or  PR  at  baseline  for  stress 
management.  Sixteen  participants  indicated  that  they  had  no  stress  in  their  life.  Of  those  who 
were  at  risk  at  baseline,  72.1%  of  those  in  the  Tx-plus  group  and  59.5%  of  those  in  the  Tx  group 
moved  to  the  A  or  M  stage. 

Figure  10.  Change  from  Risk  to  No  Risk  From  Baseline  to  Three  Months  for  Stress  Management 
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Figure  1 1 .  Stage  of  Change  Distribution  for  Depression  Prevention  at  Baseline 


About  40%  of  participants  (n=95;  40.4%)  were  in  PC,  C  or  PR  at  baseline  for  depression 
prevention.  Thirty-one  participants  indicated  that  they  had  never  been  depressed.  Of  those  who 
were  at  risk  at  baseline,  62.2%  of  those  in  the  Tx-plus  group  and  39.5%  of  those  in  the  Tx  group 
moved  to  the  A  or  M  stage. 

Figure  12.  Change  from  Risk  to  No  Risk  From  Baseline  to  3  Months  for  Depression  Prevention 


W81XWH-09-2-0138  (Final  Report)  /  page  28 


Figure  13.  Stage  of  Change  Distribution  for  Alcohol  Misuse  at  Baseline 


About  20%  of  participants  (n=46;  19.6%)  were  in  PC,  C  or  PR  at  baseline  for  alcohol 
misuse.  Notably,  76  (32.3%)  participants  indicated  that  they  had  had  no  alcohol  in  the  past  30 
days.  Given  that  almost  half  of  participants  indicated  that  they  were  in  A  or  M  for  alcohol  misuse 
(n=l  13,  48. 1%),  it’s  possible  that  some  of  those  who  did  not  consume  alcohol  in  the  past  30  days 
were  in  treatment  for  alcohol  abuse.  Of  those  who  were  at  risk  at  baseline,  60.0%  of  those  in  the 
Tx-plus  group  and  53.3%  of  those  in  the  Tx  group  moved  to  the  A  or  M  stage. 


Figure  14,  Change  from  Risk  to  No  Risk  From  Baseline  to  3  Months  for  Alcohol  Misuse 
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Figure  15.  Stage  of  Change  Distribution  for  Sleep  Problems  at  Baseline 


30  -| 


PC  C  PR  A  M 
Stage  of  Change 


Almost  two-thirds  of  participants  (n=l  5 1 ;  64.3%)  were  in  PC,  C  or  PR  at  baseline  for 
sleep  problems.  Of  those  who  were  at  risk  at  baseline,  45.5%  of  those  in  the  Tx-plus  group  and 
43.2%  of  those  in  the  Tx  group  moved  to  the  A  or  M  stage. 


Figure  16.  Change  from  Risk  to  No  Risk  From  Baseline  to  3  Months  for  Sleep  Problems 


Together  these  findings  suggest  that  while  participants  in  both  groups  were  capable  of 
multiple  behavior  change,  those  in  the  Tx-plus  group  had  accelerated  progress  through  the  stages 
of  change. 
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KEY  RESEARCH  ACCOMPLISHMENTS 


1 .  Human  use  approvals  by  both  local  IRB  and  ORP/HRPO. 

2.  Approved  modifications  to  change  PI  (from  Omizo  to  Miyahira) 

3.  Pending  modifications  to  change  PI  (from  Miyahira  to  Jordan). 

4.  Research  Project  Manager  and  Research  Assistant  hired. 

5.  Subaward  (to  Pro-Change  Behavior  Systems),  consultant  contracts  and  CRADA  signed. 

6.  Study  team  kick  off  meeting  conducted  with  Pro-Change  Behavior  Systems. 

7.  Approval  of  online  participant  informed  consent  process. 

8.  Study  Phase  1  successfully  completed:  adapting  the  web-based  CTI  to  support  text 
messaging,  tailor  the  system  interface  and  language  to  Veterans,  and  modify  the  web-based 
feedback  messages  to  cell  phone  text  messages. 

9.  Study  Phase  2  successfully  completed:  Beta  and  Usability  Testing  of  the  modified  and  tailored 
CTI. 

10.  Awarded  6  months  No  Cost  Extension  to  compensate  for  the  1 1  months  the  project  was  delayed 
before  the  current  study  team  was  involved. 

11.  Study  presented  on  invitation  to  the  National  Center  for  PTSD. 

12.  Endowment  gift  awarded  from  the  Pacific  Health  Research  and  Education  Institute  to  add  a 
supplement  study,  recruiting  additional  female  study  participants. 

13.  Study  Phase  3  successfully  completed:  235  study  participants  completed  the  study. 

14.  Study  team  data  analysis  meeting  conducted  with  Pro-Change  Behavior  Systems. 

15.  Data  analyses  completed. 

16.  Outcome  paper,  journal  manuscripts,  and  conference  presentations  prepared. 
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REPORTABLE  OUTCOMES 


Publications 

1.  Jordan,  P.  J.,  Lid,  V.,  &  Evers,  K.  E.  (2012).  Cell  phone-enhanced  expert  systems  to 
promote  smoking  cessation  in  veterans.  (Manuscript  in  preparation.) 

2.  Jordan,  P.  J.,  Lid,  V.,  &  Evers,  K.  E.  (2012).  Gender  differences  in  smoking  cessation 
outcomes  in  veterans.  (Manuscript  in  preparation.) 

Presentations 

1.  Jordan,  P.  J.,  Lid,  V.,  &  Evers,  K.  E.  (2012).  Cell  phone-enhanced  expert  systems  to 
promote  smoking  cessation  in  veterans.  Paper  accepted  for  presentation  at  the  16th  Annual 
International  meeting  and  Exposition  of  the  American  Telemedicine  Association  in  San  Jose, 
CA,  April  29-May  1,2012. 

2.  Jordan,  P.  J.,  King,  L.  A.,  Lid,  V.,  Evers,  K.  E.,  &  Nigg,  C.  R.  (2012).  Stage  of  change  for 
multiple  behaviors  in  veterans  with  and  without  PTSD.  Poster  accepted  for  presentation  at 
the  33rd  Annual  Meeting  and  Scientific  Sessions  of  the  Society  of  Behavioral  Medicine,  New 
Orleans,  LA,  April  11-14,  2012. 

3.  Nigg,  C.,  Huang,  Y.,  Jordan,  P.  J.,  Burke,  K.,  Kawasaki,  M.,  Evers,  K.,  King,  L.,  Daly,  S., 
&  Spira,  J.  (2012).  Using  focus  groups  with  veterans  to  identify  issues  to  adapt  a 
computerized  tailored  intervention  to  Address  PTSD  related  behavioral  risk  factors.  Paper 
accepted  for  presentation  at  IADIS  International  Conference  e-Society  2012,  Berlin, 
Gennany,  March  10-13,  2012. 

4.  Jordan,  P.  J.,  King,  L.,  Lid,  V.  (2011).  A  web-based  methodology  for  promoting  health 
behavior  change  in  Veterans  with  PTSD-related  comorbidities.  Presentation  to  the  National 
Center  for  PTSD,  PTSD  Research  Group,  18-August-201 1,  via  videoteleconference. 

Awards 

Endowment  gift  awarded  from  the  Pacific  Health  Research  and  Education  Institute  to  add  a 

supplement  study,  recruiting  additional  female  study  participants. 
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CONCLUSION 


In  conclusion,  the  evidence-based  CTI  is  generally  applicable  to  recent  Veterans 
returning  from  Iraq  and  Afghanistan.  Some  minor  modifications  can  be  made  to  further  tailor  the 
programs  to  Veterans’  specific  needs.  As  a  group,  Veterans  have  a  distinctive  sociodemographic 
profile  (Koepsell,  2002).  Examining  the  mental  health  effects  in  recent  Veterans  returning  from 
Iraq  and  Afghanistan  has  been  of  increasing  importance,  particularly  since  research  has  shown 
that  deployment  and  exposure  to  combat  result  in  increased  risk  of  PTSD,  major  depression, 
substance  abuse,  alcohol  misuse,  functional  impairment,  and  increased  use  of  health  care 
services  (Hoge,  et  ah,  2006;  Hoge,  et  ah,  2007). 

This  study  was  able  to  demonstrate  that  the  TTM  is  an  appropriate  model  to  promote 
multiple  behavior  change  in  Veterans.  Furthermore,  the  addition  of  tailored  text  messages 
increased  the  effect  of  the  smoking  cessation  intervention  (CTI)  and  improved  quit  rates  in  the 
Tx-plus  group  to  levels  that  are  unprecedented  in  the  literature.  Additionally,  individuals  who 
were  at  risk  for  other  health  behaviors  progressed  to  A  or  M  without  intervention. 

For  Veterans  who  are  not  ready  for  in-person  psychotherapy,  have  practical  issues 
attending  therapy  sessions,  or  fear  stigma  associated  with  seeking  mental  health  care,  this 
Veteran-tailored,  motivational  enhancement  CTI  may  provide  unique  benefits.  In  addition, 
Veterans  who  smoke  and  have  comorbid  behavioral  risk  factors  may  find  this  CTI  sufficient  to 
resolve  their  problems.  In  this  sense,  the  adapted  CTI  will  make  evidence-based  mental  health 
care  more  accessible  to  Veterans  in  need  without  adding  concerns  about  stigma  and  costs. 
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Table  1.  Eligibility  Criteria  for  Focus  Group  Participants 


Eligible  Individuals _ Ineligible  Individuals 


1.  Veterans  age  18+  years 

2.  Iraq  or  Afghanistan  service  experience 
preferred 

3.  Computer  literacy  at  the  beginner  level 
or  higher 

4.  Cigarette  smoker 

5.  Mild  to  moderate  PTSD  symptoms 


1 .  History  of  mania,  schizophrenia,  other 
psychoses,  or  active  substance  use 

2.  Special  medical  conditions  that  may 
prevent  engagement  with  the  CTI  program, 
such  as  history  of  significant  head  injury 

3.  Suicidal  ideation 

4.  Severe  PTSD  symptoms 

5.  Severe  depressive  symptoms 


Note.  PTSD=posttraumatic  stress  disorder.  CTI=computerized,  tailored  intervention. 


W8 1 XWH-09-2-0 138  (Final  Report)  /  page  41 


Table  2.  Demographics  Collected  on  Focus  Group  Participants  (n=9) 


Demographics 

(N=9) 

Mean  Age  in  Years  (Range) 

54.4  (45-68) 

Branch 

Air  Force 

3  (33.3%) 

Army 

4  (44.4%) 

Marines 

2  (22.2%) 

Navy 

0  (0%) 

Mean  Months  Served  (Range) 

66.11  (8-240) 

Ethnicity 

African  American 

1  (11.1%) 

Hispanic/Latino 

0  (0%) 

Pacific  Islander 

5  (55.6%) 

Caucasian 

3  (33.3%) 

Marital  Status 

Never  Married 

3  (33.3%) 

Married 

1  (11.1%) 

Separated 

2  (22.2%) 

Divorced 

3  (33.3%) 

Living  with  Partner 

2  (22.2%) 

Has  Children 

3  (33.3%) 

Education 

Some  High  School 

1  (11.1%) 

High  School/GED 

5  (55.6%) 
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Table  3.  Demographics  Collected  on  Usability  Interview  Participants  (n=5) 


Demographics 

(N=5) 

Mean  Age  in  Years  (Range) 

50.0  (36-63) 

Branch 

Anny 

2  (40%) 

Navy 

3  (60%) 

Mean  Months  Served  (Range) 

50.0  (24-94) 

Ethnicity 

Asian 

1  (20%) 

Native  American/ 

1  (20%) 

Alaskan  Native 

White/Caucasian 

3  (60%) 

Marital  Status 

Never  Married 

2  (40%) 

Divorced 

3  (60%) 

Living  with  Partner 

0  (0%) 

Education 

Some  College 

3  (60%) 

Graduate  Degree 

2  (40%) 
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Table  4.  Baseline  Participant  Demographics  (n=446) 


ASSIGNED  GROUP 

Demographics 

Treatment-Plus 

Treatment 

Total 

Gender* 

N 

% 

N 

% 

N 

% 

Male 

128 

61.0 

190 

50.4 

247 

55.4 

Female 

82 

39.0 

117 

49.6 

199 

44.6 

Ethnicity 

White,  non-Hispanic 

160 

76.2 

173 

73.3 

333 

74.7 

Black,  non-Hispanic 

15 

7.1 

28 

11.9 

43 

9.6 

Asian  American 

2 

1.0 

8 

3.4 

10 

2.2 

Native  Hawaiian,  Other  Pacific 

Islander 

5 

2.4 

3 

1.3 

8 

1.8 

American  Indian,  Alaska  Native 

5 

2.4 

2 

0.8 

7 

1.6 

Hispanic 

20 

9.5 

17 

7.2 

37 

8.3 

Other 

3 

1.4 

5 

2.1 

8 

1.8 

Marital  Status 

Single,  never  married 

40 

19.0 

48 

20.3 

88 

19.7 

Living  with  a  partner 

19 

9.0 

24 

10.2 

43 

9.6 

Married 

95 

45.2 

108 

45.8 

203 

45.5 

Separated 

8 

3.8 

9 

3.8 

17 

3.8 

Divorced 

43 

20.5 

44 

18.6 

87 

19.5 

Widowed 

5 

2.4 

3 

1.3 

8 

1.8 

Education 

Less  than  HS 

3 

1.4 

2 

0.8 

5 

1.1 

High  School 

31 

14.8 

32 

13.6 

63 

14.1 

Some  College 

117 

55.7 

131 

55.5 

248 

55.6 

College  Graduate 

49 

23.3 

62 

26.3 

111 

24.9 

Postgraduate 

10 

4.8 

89 

3.8 

19 

4.3 

Age 

M  (SD) 

38.54 

(11.6) 

37.18 

(10.4) 

37.82 

(11.0) 

Range 

20-68 

20-65 

20-68 

*  p  <  .05 
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Table  5.  Baseline  Military  History  (n=446) 


ASSIGNED  GROUP 

Military  History 

Treatment-Plus 

Control 

Total 

Military  Service  Branch 

N 

% 

N 

% 

N 

% 

Anny 

95 

45.2 

96 

40.7 

191 

42.8 

Marines 

22 

10.5 

32 

13.6 

54 

12.1 

National  Guard 

12 

5.7 

13 

5.5 

25 

5.6 

Navy 

25 

11.9 

48 

20.3 

73 

16.4 

Airforce 

22 

10.5 

20 

8.5 

42 

9.4 

Coast  Guard 

1 

0.5 

1 

0.4 

2 

0.4 

Other 

1 

0.5 

3 

1.3 

4 

0.9 

Combination 

32 

15.2 

23 

9.7 

55 

12.3 

Rank  at  Discharge 

Enlisted 

103 

49.0 

120 

50.8 

223 

50.0 

Senior  Enlisted 

100 

47.6 

114 

48.3 

214 

48.0 

Officer 

7 

3.3 

2 

0.8 

9 

2.0 

Years  in  Military 

M  (SD) 
Range 

9.20 

0.5-35.9 

(7.22) 

8.37 

0.6-33.3 

(6.51) 

8.76 

0.5-35.9 

(6.86) 
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Table  6.  Baseline  Smoking  History  (n=446) 


ASSIGNED  GROUP 


Smoking  History 

Treatment-Plus 

Control 

Total 

Started  Smoking 

N 

% 

N 

% 

N 

% 

Before  joining  service 

130 

55.1 

253 

56.7 

During  boot  camp 

123 

58.6 

24 

10.2 

47 

10.5 

After  boot  camp 

23 

11.0 

50 

21.2 

94 

21.1 

After  deployment 

18 

8.6 

28 

11.9 

46 

10.3 

After  discharge 

2 

1.0 

4 

1.7 

6 

1.3 

Number  of  daily  cigarettes 

10  or  less 

27 

12.9 

32 

13.6 

59 

13.2 

11-20 

143 

68.1 

152 

64.4 

295 

66.1 

21-30 

25 

11.9 

33 

14.0 

58 

13.0 

31+ 

15 

7.1 

19 

8.1 

31 

7.6 

Number  of  quit  attempts 

0 

11 

5.2 

11 

4.7 

22 

4.9 

1-3 

89 

42.3 

97 

41.1 

186 

41.7 

4-5 

46 

26.7 

66 

27.9 

122 

27.4 

6-9 

11 

5.3 

23 

9.8 

34 

7.6 

10  or  more 

43 

20.5 

39 

16.5 

82 

18.4 

In  addition  to  cigarettes, 

also  smoking/using... 

Pipe 

10 

4.8 

14 

5.9 

24 

5.4 

Cigar 

38 

18.1 

45 

19.1 

83 

18.6 

Cheroot 

3 

1.4 

1 

0.4 

4 

0.9 

Hookah 

16 

7.6 

14 

5.9 

30 

6.7 

Bidis 

1 

.5 

2 

0.8 

3 

0.7 

Kreteks 

3 

1.4 

10 

4.2 

13 

2.9 

Snuff 

6 

2.9 

5 

2.1 

11 

2.5 

Chewing  tobacco 

11 

5.2 

10 

4.2 

21 

4.7 

Dip 

17 

8.1 

13 

5.5 

30 

6.7 

Snus 

10 

4.8 

12 

5.1 

22 

4.9 

Spouse/partner  smokes 

No 

84 

40.0 

104 

44.1 

188 

42.2 

Yes 

76 

36.2 

73 

30.9 

149 

33.4 

N/A 

50 

23.8 

59 

25.0 

109 

24.4 
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Table  7.  Baseline  Scale  Scores  (n=446) 


ASSIGNED  GROUP 

Smoking  Scales 

Treatment-Plus 

Control 

Total 

FNDS  Score 

Mean  (SD) 
Range 

4.9  (2.1) 

0-10 

4.7  (2.2) 

0-10 

4.8  (2.1) 

0-10 

QSU  Scale  Score 

Mean  (SD) 
Range 

43.7  (15.0) 

10-70 

43.6  (14.2) 

10-70 

43.7  (14.6) 

10-70 

Stage  of  Change  for  Smoking 
Cessation 

Precontemplation 

Contemplation 

Preparation 

27  12.9 

129  61.4 

54  25.7 

26  11.0 

152  64.4 

28  24.6 

53  11.9 

281  63.0 

112  25.1 

Note.  FNDS  =  Fagerstrom  Nicotine  Dependence  Scale.  QSU  =  Questionnaire  for  Smoking 
Urges. 
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Table  8.  Results  for  Quantitative  Evaluation  Questions  (n=235) 


Evaluation  Questions  (All  Participants) 

Rating  (%) 

Strongly 
Disagree 
and  Disagree 

No 

Opinion 

Agree 

and 

Strongly 

Agree 

The  program  was  easy  to  use. 

2.55 

5.11 

92.34 

The  questions  were  easy  to  understand. 

1.70 

6.38 

91.91 

The  personal  feedback  was  easy  to  understand. 

1.70 

6.38 

91.91 

I  like  the  overall  look  of  the  program. 

3.83 

7.23 

88.94 

The  program  was  made  for  people  my  age. 

1.28 

15.74 

82.98 

The  personal  feedback  was  interesting. 

2.55 

11.91 

85.53 

I  enjoyed  using  the  program. 

4.68 

11.06 

84.26 

The  program  gave  me  new  things  to  think  about.*3 

3.40 

8.94 

87.66 

The  program  could  help  me  be  healthier. 

4.68 

7.23 

88.09 

The  program  made  me  consider  changing  my  smoking  habits.* 

5.11 

7.23 

87.66 

I  would  like  to  use  this  program  again  in  the  future  to  see  if/how  I’ve 
changed.*3 

4.68 

13.62 

81.70 

I  would  recommend  this  program  to  a  friend.*3 

2.55 

11.91 

85.53 

Overall,  I  feel  this  program  helped  me  achieve  the  results  I  wanted.* 

5.53 

26.38 

68.09 

I  would  like  to  be  able  to  use  this  program  more  often. 

5.96 

22.98 

71.06 

I  found  the  program  too  complex.  *b 

70.64 

11.91 

17.45 

I  think  the  support  of  a  technical  person  would  have  helped  me  better 
use  this  program.  *b 

52.77 

29.36 

17.87 

I  thought  this  program  was  too  confusing  to  use.* 

79.57 

11.91 

8.51 

I  think  that  most  people  could  learn  to  use  this  program  very  quickly. 

1.70 

10.64 

87.66 

I  found  the  program  very  difficult  to  use. 

81.70 

10.64 

7.66 

I  felt  very  confident  using  the  program. 

1.28 

14.04 

84.68 

I  need  to  leam  a  lot  of  things  before  I  could  benefit  from  this  program. 

64.68 

18.72 

16.60 

Note.  *  p  <  .05  by  stage  of  change.  a=precontemplation  <  contemplation  only.  b=precontemplation  < 
action  only.  c=precontemplation  <  contemplation,  preparation  and  action.  d=precontemplation  < 
contemplation  and  action. 
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Table  9.  Results  for  Evaluation  Questions  of  text  Messages  (n=T  19) 


Evaluation  Questions  (Tx-plus  group  only) 

Rating  (%) 

Strongly 

Disagree 

and  Disagree 

No 

Opinion 

Agree  and 
Strongly 
Agree 

The  text  messages  were  easy  to  understand. 

1.68 

7.56 

90.76 

The  text  messages  arrived  at  times  when  it  was  good  for  me  to 
receive  them. 

17.65 

10.92 

71.43 

The  text  messages  helped  me  move  towards  quitting 
smoking.*3 

9.24 

18.49 

72.27 

Getting  text  messages  was  annoying. 

52.94 

22.69 

24.37 

I  would  have  liked  more  text  messages  from  the  program. 

34.45 

31.93 

33.61 

I  didn’t  read  the  text  messages  from  the  program. 

84.03 

6.72 

9.24 

I  found  the  text  messages  difficult  to  understand. 

86.55 

5.04 

8.40 

I  liked  receiving  the  text  messages  from  the  program. 

9.24 

21.85 

68.91 

The  texting  reinforced  what  I  learned  from  the  online 
program. 

6.72 

13.45 

79.83 

The  text  messages  gave  me  new  things  to  think  about  in  terms 
of  my  smoking. 

10.92 

10.08 

78.99 

Note.  *  p  <  .05  by  stage  of  change.  a=precontemplation  <  contemplation,  preparation  and  action. 
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Table  10.  Demographics  for  Completers  ( n=23 5 ) 


ASSIGNED  GROUP 

Demographics 

Treatment-Plus 

Treatment 

Total 

Gender* 

N 

% 

N 

% 

N 

% 

Male 

72 

60.5 

50 

43.1 

122 

51.9 

Female 

47 

39.5 

66 

56.9 

113 

48.1 

Ethnicity 

White,  non-Hispanic 

88 

73.9 

79 

68.1 

167 

71.1 

Black,  non-Hispanic 

10 

8.4 

18 

15.5 

28 

11.9 

Asian  American 

1 

0.8 

7 

6.0 

8 

3.4 

Native  Hawaiian,  Other  Pacific 

Islander 

2 

1.7 

1 

0.9 

3 

1.3 

American  Indian,  Alaska  Native 

4 

3.4 

1 

0.9 

5 

2.1 

Hispanic 

11 

9.2 

8 

6.9 

19 

8.1 

Other 

3 

2.5 

2 

1.7 

5 

2.1 

Marital  Status 

Single,  never  married 

32 

26.9 

31 

26.7 

63 

26.8 

Living  with  a  partner 

12 

10.1 

15 

12.9 

27 

11.5 

Married 

46 

38.7 

52 

44.8 

98 

41.7 

Separated 

4 

3.4 

2 

1.7 

6 

2.6 

Divorced 

23 

19.3 

15 

12.9 

38 

16.2 

Widowed 

2 

1.7 

1 

0.9 

3 

1.3 

Education 

Less  than  HS 

2 

1.7 

1 

0.9 

3 

1.3 

High  School 

16 

13.4 

14 

12.1 

30 

12.8 

Some  College 

70 

58.8 

59 

50.9 

129 

54.9 

College  Graduate 

25 

21.0 

37 

31.9 

62 

26.4 

Postgraduate 

6 

5.0 

5 

4.3 

11 

4.7 

Age 

M  (SD) 

37.0 

(11.5) 

34.7 

(9.4) 

35.9 

(10.5) 

Range 

22-65 

20-64 

20-65 

*£<.05 
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Table  1 1 .  Baseline  Scores  for  Completers  (n=235 ) 


FNDS  Score 

Mean  (SD) 
Range 

4.9  (2.2) 

0-10 

4.7  (2.1) 

0-9 

4.8  (2.2) 

0-10 

QSU  Scale  Score 

Mean  (SD) 
Range 

45.2  (14.0) 

44.2  (14.3) 

10-70 

44.7  (14.1) 

10-70 

Stage  of  Change  for  Smoking 
Cessation 

Precontemplation 

Contemplation 

Preparation 

15  12.6 

66  55.5 

38  31.9 

17  14.7 

70  60.3 

29  25.0 

32  13.6 

136  57.9 

67  28.5 

Note.  FNDS  =  Fagerstrom  Nicotine  Dependence  Scale.  QSU  =  Questionnaire  for  Smoking  Urges. 
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Table  12.  Comparison  of  TTM  Constructs  Between  Groups  Across  Timcpoints. 


Tx  Group 

(mean  [SD])  (n=l 

116) 

Tx-Plus 

(mean  [SD])  (n=l 

19) 

B/L 

T1 

T3 

B/L 

T1 

T3 

CONSTRUCT 

Pros 

22.7 

22.2 

22.  lb 

22.3 

21.8 

2L35 

[5.31 

[5.71 

[5.81 

T5.21 

[5.71 

(5.8) 

Cons 

20.3 

19.5 

19.1 

21.2 

20.8 

18.6 

[5.11 

[5.81 

T6.01 

T5.31 

[5.81 

(6.2) 

DCBL 

2.4 

2.7 

3.7 

1.11 

1.0 

2.7 

re.ii 

T6.61 

[7.31 

[6.61 

[7.11 

(8.5) 

SE 

20.9 

23.8 

27.8 

20.4 

23.0 

21 A 

rv.si 

T8.21 

T8.31 

T7.31 

T8.21 

[8.71 

CR 

9.0 

10.4 

11.3 

9.7 

10.3 

11.2 

T3.41 

T3-21 

T3-31 

T2.91 

T3.41 

[3-21 

DR 

9.0 

9.0 

9.5 

8.2 

8.9 

9.0 

[3.21 

T3.il 

T3.31 

T2.81 

[3-21 

T3.41 

ER 

6.8 

6.2 

R65 

5.6 

4.4 

NO5 

T2.31 

T2.31 

T3-61 

T2.71 

[1.91 

T2-21 

SL 

8.6 

9.6 

10.5 

8.2 

9.3 

9.6 

T3-31 

T3.il 

T3.il 

T3.ll 

T2.91 

[3-11 

SR 

9.1 

9.7 

9.9 

8.9 

9.6 

9.9 

[3-51 

T3.il 

[3-21 

T3.il 

T2.81 

[3-31 

CC 

8.4 

9.5 

11.4 

8.3 

9.9 

11.1 

T3-31 

T3.31 

T3-31 

T3.31 

T3.31 

[3.31 

SC 

7.9 

8.6 

9.4 

7.3 

8.7 

8.8 

[3.51 

[3-31 

[3-31 

T3-21 

T3.il 

[3-31 

HR 

fccP 

10.9 

11.6 

8A5 

9.8 

11.0 

r  3.2i 

T3-41 

T3-41 

T3-41 

T3.61 

[3-71 

RM 

8.7 

9.6 

IL05 

8.5 

9.5 

T3.il 

T3.il 

[3-11 

T3.il 

[3-11 

[3-31 

SO 

10.9 

11.1 

11.05 

10.7 

10.7 

11.6 

T2.81 

T2.91 

T3-61 

T3.ll 

T3.21 

[3.21 

Note.  B/L=baseline.  Tl=One -month  follow-up  T3=three-month  follow-up.  DCBL=decisional  balance 
(pros  minus  cons).  SE=self-efficacy.  CR=consciousness  raising.  DR=dramatic  relief.  ER=environmental 
reevaluation.  SL=self-liberation.  SR=self-reevaluation.  CC=counter-conditioning.  SC=stimulus  control. 
SS=helping  relationships.  RM=reinforcement  management.  SO=social  liberation. 
a=significant  difference  within  group.  b=significant  difference  between  group  (p  <  .05). 
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Table  13.  Comparison  of  Scale  Scores  Between  Groups  Across  Timepoints. 


Tx  Group 

(mean  [SD])  (n=l 

[16) 

Tx-Plus 

(mean  [SD])  (n=119) 

B/L 

T1 

T3 

B/L 

T1 

T3 

SCALE 

QSU 

44.2 

34. 5a 

25.5 

45.2 

30. 1 a 

25.5 

[14.31 

[15.51 

[14.41 

[14.01 

[14.81 

(14.4) 

FNDS 

4.7 

3.4 

3.3 

4.9 

3.6 

3.5 

[2.11 

[2.11 

(2.2) 

[2.21 

r2.2i 

(2.1) 

Note.  B/L=baseline.  Tl=One -month  follow-up  T3=three-month  follow-up.  QSU=questionnaire  on 
smoking  urges.  FNDS=Fagerstrom  nicotine  dependence  scale.  a=sign  ill  cant  difference  between  group 
(P  <  .05). 
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Table  14.  TTM  Constructs  Across  Time  by  Stage  of  Change  Within  and  Between  Groups 


Tx  Group  (mean  [SD])  (n=l  16) 

Tx-Plus  (mean  [SD])  (n=l  19) 

B/L  Stage  of  Change 

T3  Stage  of  Chan: 

se 

B/L  Stage  of  Change 

T3  Stage  of  Chan; 

ge 

PC 

(n=17) 

c 

(n=70) 

PR 

(n=29) 

PC 

(n=8) 

c 

(n=38) 

PR 

(n=32) 

A 

(n=38) 

PC 

(n=15) 

c 

(n=66) 

PR 

(n=38) 

PC 

(n=7) 

c 

(n=37) 

PR 

(n=24) 

A 

(n=51) 

Pros3 

18.9° 

[1.2] 

23.6 

[0.6] 

22.6 

[0.9] 

18.9 

[5.0] 

22.0 

[6.1] 

24.7 

[4.7] 

25.6 

[5.3] 

16. 8C 
[1.3] 

22.1 

[0.6] 

24.8 

[.0.8] 

13.6 

[5.0] 

21.8 

[5.1] 

22.9 

[5.7] 

25.7 

[4.1] 

Cons 

19.4 

[1.3] 

20.2 

[0.6] 

20.8 

[1.0] 

18.2 

[7.4] 

20.5 

[5.6] 

17.6 

[5.9] 

22.1 

[7.0] 

20.2 

[1.3] 

21.9 

[0.6] 

20.4 

[0.8] 

17.7 

[8.1] 

19.6 

[5.4] 

18.0 

[7.1] 

20.9 

[7.1] 

DCBL 

-0.5 

[7.3] 

3.4C 

[5.9] 

1.8 

[5.1] 

0.63 

[3.5] 

1.5 

[6.5] 

7.1 

[7.71 

3.5 

[6.3] 

-3.4 

[5.6] 

0.2C 

[6.1] 

4.4 

[6.3] 

-4.1 

[6.4] 

2.4 

[6.9] 

5.0 

[10.0] 

4.8 

[6.9] 

SE3 

N/A 

19.4 

[7.9] 

24.6 

[6.1] 

N/A 

22.0 

[6.1] 

28.2 

[6.2] 

32.9 

[8.4] 

N/A 

17.9 

[6.2] 

24.6 

[7.2] 

N/A 

20.5 

[6.6] 

26.3 

[7.6] 

32.9 

[6.7] 

CR 

6.0 

[2.41 

N/A 

10.8 

[2.51 

6.9 

[2.91 

10.3 

[2.71 

11.6 

[2.71 

12.2 

[3.11 

7.3 

[2.91 

N/A 

10.6 

[2.3] 

7.1 

[2.71 

11.3 

D.21 

10.9 

r3.oi 

11.9 

r3.oi 

DR 

6.6 

[2.4] 

9.6 

[3.11 

N/A 

7.1 

[2.9] 

10.0 

[3.2] 

11.0* 

N/A 

6.1 

[2.1] 

8.7 

[2.6] 

N/A 

5.3 

[2.7] 

9.7 

[3.2] 

9.0 

[0.0] 

N/A 

ER 

6.9 

[2.3] 

N/A 

N/A 

8.1 

[3.7] 

N/A 

N/A 

N/A 

5.6 

[2.7] 

N/A 

N/A 

5.0 

[2.2] 

N/A 

N/A 

N/A 

SL3 

4.9 

[2.1] 

8.6 

[3.2] 

10.7 

[2.3] 

7.0 

[2.8] 

10.0 

[2.8] 

11.9 

[2.8] 

N/A 

5.7 

[3.1] 

7.8 

[2.7] 

10.1 

[2.9] 

5.3 

[2.2] 

9.6 

[2.7] 

11.0 

[2.7] 

N/A 

SR 

5.9 

[2.6] 

9.9 

[3.21 

N/A 

7.4 

[3.5] 

10.4 

[2.9] 

11.0* 

N/A 

6.0 

[2.4] 

9.5 

[2.8] 

N/A 

6.1 

[1.9] 

10.7 

[3.1] 

8.5 

[0.7] 

N/A 

CC3 

5.3 

[2.1] 

8.4 

[3.4] 

10.3 

[2.2] 

7.4 

[2.6] 

9.9 

[3.2] 

11.5 

[2.8] 

13.5 

[2.1] 

5.8 

[2.9] 

7.9 

[3.0] 

10.0 

[3.0] 

5.4 

[2.6] 

9.5 

[3.0] 

11.1 

[3.1] 

13.0 

[2.1] 

SC3 

4.6 

[2.0] 

7.6 

[3.51 

10.3 

[2.4] 

6.1 

[3.4] 

8.8 

[2.9] 

10.9 

[3.1] 

N/A 

5.5 

[2.6] 

6.9 

[3.0] 

8.6 

[3.3] 

5.1 

[2.3] 

8.8 

[3.2] 

10.0 

[3.0] 

N/A 

HR 

N/A 

N/A 

10.2b 

[3.2] 

N/A 

11.3 

[1.5] 

10.5 

[3.6] 

12.5 

[3.2] 

N/A 

N/A 

8.4b 

[3.4] 

N/A 

9.3 

[3.1] 

10.4 

[4.0] 

11.4 

[3.6] 

RM 

5.6 

[2.0] 

8.8 

[3.0] 

10.6 

[2.2] 

6.8 

[2.9] 

9.7 

[2.5] 

11.1 

[2.7] 

13.0 

[2.5] 

5.7 

[2.4] 

8.3 

[3.0] 

9.9 

[2.6] 

4.9 

[1.8] 

9.3 

[2.7] 

10.6 

[2.9] 

12.4 

[2.5] 

SO 

N/A 

N/A 

10.9 

[2.8] 

N/A 

10.5 

[3.9] 

10.5 

[3.5] 

11.6 

[3.6] 

N/A 

10.7 

[3.1] 

N/A 

N/A 

12.0 

[2.0] 

11.1 

[3.4] 

11.8 

[3.2] 

Note.  B/L=baseline.  Tl=One-month  follow-up  T3=three-month  follow-up.  DCBL=decisional  balance  (pros  minus  cons).  SE=self-efficacy.  CR=consciousness 
raising.  DR=dramatic  relief.  ER=environmental  reevaluation.  SL=self-liberation.  SR=self-reevaluation.  CC=counter-conditioning.  SC=stimulus  control. 
SS=helping  relationships.  RM=reinforcement  management.  SO=social  liberation. a=significant  difference  within  group  between  stages.  b=significant  difference 
between  group,  p<.05.  c=significant  interaction  effect,  p<.05.  N/A=not  assessed.  *  no  standard  deviation,  n=l. 
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APPENDIX  A:  LIST  OF  PERSONNEL 


Principal  Investigator  (15%): 
Co-Principal  Investigator  (5%): 
Co-Investigator  (5%): 

Research  Project  Manager  (100%): 
Research  Assistant  (35%): 


Patricia  J.  Jordan,  Ph.D. 
Julia  Whealin,  Ph.D. 
James  Spira,  Ph.D. 

Viil  Lid,  M.S. 

Stacy  Daly,  B.S. 
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APPENDIX  B:  STUDY  PROGRESS 
Year  One 


During  the  first  year  of  the  project,  several  delays  and  personnel  changes  impacted  the 
anticipated  progress.  A  summary  timeline  is  presented  below. 

1 .  August  3 1 st  2009:  The  project  was  awarded. 

2.  November  19th  2009:  Protocol  was  submitted  to  VA  PIHCS. 

3.  December  17th  2009:  Approval  of  the  protocol  was  received  from  VA  PIHCS  IRB. 

4.  February  3ld  2010:  Official  change  in  PI  from  Dr.  Reese  Omizo  to  Dr.  Sarah  Miyahira 
received. 

5.  March  1st  2010:  Name  change  of  contracting  organization  from  VARECORP  to  PHREI 
approved. 

6.  April  6th  2010:  Approved  protocol  sent  to  ORP/HRPO. 

7.  May  1 1th  2010:  Human  Use  approvals  received  from  HRPO. 

8.  July  23ld  2010:  Request  change  in  PI  from  Dr.  Sarah  Miyahira  to  Dr.  Patricia  J.  Jordan. 

9.  August  10th  2010:  Budget  Reallocation  request  submitted. 

Year  Two 


During  the  second  year  of  the  project,  all  team  members  were  hired,  study  Phases  1  and  2  were 

completed,  and  Phase  3  was  commenced.  A  summary  timeline  is  presented  below. 

10.  September  2010:  Viil  Lid  was  hired  as  Research  Project  Manager. 

11.  October  2010:  Subaward  contract  between  Pro-Change  and  PHREI  was  signed. 

12.  October  2010:  Contract  for  professional  services  to  be  provided  by  Dr.  James  Prochaska  was 
completed. 

13.  October  21st  -  22nd  2010:  Kick-off  Meeting  for  the  study  team  was  held  at  VAPIHCS.  The 
meeting  was  attended  by  all  local  team  members,  as  well  as  Dr.  Kerry  Evers  and  Dr.  Jim 
Prochaska  from  Pro-Change  in  Rhode  Island. 

14.  November  2010:  The  project  IT  Coordinator  position  was  eliminated  as  all  IT  needs  for  the 
project  are  handled  by  the  Pro-Change  subaward. 

15.  November  2010:  Stacy  Daly  was  hired  as  a  Research  Assistant. 

16.  November  2010:  All  team  members  completed  VAPIHCS  IRB  Informed  Consent  training. 

17.  November  2010:  Budget  Reallocation  was  approved  by  the  COR  and  CS. 

18.  December  2010:  An  internet  domain  with  security  certificates  (www.txtresearch.org)  was 
acquired  to  host  the  participant  online  interface  to  the  web-based  CTI  system. 

19.  December  2010:  A  smart  phone  with  telephone,  text  messaging,  and  internet  subscription 
was  purchased  in  order  to  use  for  focus  group  recording,  cell  phone-based  system  testing,  and 
participant  recruitment. 
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20.  January  201 1 :  A  Focus  Group  with  Veterans  was  conducted  to  gather  information  and 
feedback  to  inform  adaptation  and  tailoring  of  the  web-based  CTI  and  integrated  cell  phone 
text  messages. 

21.  January  2011:  Study  Phase  1  started.  The  web-based  CTI  was  adapted  to  support  text 
messaging,  the  system  interface  and  language  was  tailored  to  Veterans,  and  the  web-based 
feedback  messages  were  modified  to  cell  phone  text  messages. 

22.  February  2011:  Several  amendments  to  the  protocol  and  other  project  documents  were 
approved  at  a  full  IRB  board  meeting.  These  amendments  allow  us  to  move  the  informed 
consenting  process  online  and  automate  the  sending  of  cell  phone  text  messages  to 
participants. 

The  process  of  developing  and  approving  these  amendments  started  in  October  2010  and 
involved  careful  studying  of  VA  policies  and  several  meetings  and  coordination  with 
VAPIHCS  Privacy  Officer,  VAPIHCS  Infonnation  Security  Officer,  VAPIHCS  IRB 
Coordinator,  and  the  Pro-Change  technology  design  and  development  team  to  discuss  options 
and  solutions. 

As  far  as  we  know  the  VA  IRB  approval  of  online  infonned  consent  is  unprecedented  in 
the  nationwide  VA  system,  and  is  a  noteworthy  accomplishment  by  our  team.  Through  our 
efforts  we  believe  we  improved  human  subject  protection,  data  validity,  and  research 
practicability  and  effectively. 

23.  February  201 1 :  A  CRADA  between  Pro-Change  and  PHREI  was  signed. 

24.  March  2011:  Study  Phase  2  started.  Beta  and  Usability  Testing  of  the  modified  and  tailored 
web-  and  cell  phone-based  CTI. 

25.  March/ April  2011:  Minor  amendments  to  the  protocol  and  other  project  documents  were 
approved  by  the  IRB. 

26.  April  2011:  Study  Phase  3  launched.  The  web-  and  cell  phone-based  CTI  was  launched  on 
April  27th  2011,  and  from  through  August  2011  various  initiatives  were  commenced  in  order 
to  recruit  study  participants. 

27.  August  2011:  Application  for  a  six  month  No  Cost  Extension  for  the  project  was  approved. 
The  POP  end  date  is  March  31,  2012. 

28.  August  18th  2011:  The  Dr.  Jordan  and  Ms.  Lid  were  invited  to  present  the  project  at  a  Video 
Conference  for  the  National  Center  for  PTSD. 

29.  August  2011:  The  project  team  was  awarded  an  endowment  gift  from  the  Pacific  Health 
Research  and  Education  Institute  to  add  a  supplement  study  recruiting  additional  women 
participants  to  offset  the  gender  imbalance  in  the  Veteran  population  so  that  we  can  do  a 
gender  comparison  with  statistical  power. 

Year  Three 

During  the  final  6  months  of  the  project  study  Phase  3  was  completed,  study  data  were  analyzed, 

and  the  hypotheses  were  tested.  A  summary  timeline  is  presented  below. 

30.  September  8th  201 1 :  a  protocol  amendment  was  approved  by  VAPIHCS  IRB  to  allow  for 
recruiting  additional  participants  for  the  supplement  study. 

31.  October  13th  201 1:  the  second  annual  continuing  review  was  approved  by  VAPIHCS  IRB. 

32.  November  1 1th  2011:  Recruitment  of  study  participants  ended. 

33.  January  2012:  An  abstract  by  Dr.  Patricia  Jordan  was  accepted  for  presentation  at  SBM’s  annual 
meeting  &  scientific  sessions,  April  11-14  2012,  New  Orleans. 
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34.  January  2012:  Abstracts  by  Dr.  Patricia  Jordan  and  Project  Manager  Viil  Lid  were  accepted  for 
presentation  at  ATA’s  annual  international  meeting  &  expo,  April  29  -  May  1  2012,  San  Jose. 

35.  February  1st  2012:  Study  Phase  3  ended.  The  web-  and  cell  phone-based  CTI  was  closed  and 
the  data  collection  ended.  Data  cleaning  and  analysis  commenced. 

36.  February  27th  -  28th  2012:  Post  Intervention  Assessment  Meetings  for  the  study  team  were 
held  at  VAPIHCS  and  PHREI.  The  meeting  was  attended  by  all  local  team  members,  as  well 
as  Dr.  Kerry  Evers  and  Julie  Padula  from  Pro-Change  in  Rhode  Island. 

37.  March  15,  2012:  Abstract  entitled:  “Can  text  messaging  increase  smoking  cessation  rates  in 
Veterans?”  was  submitted  by  Dr.  Jordan  to  the  International  Society  for  Traumatic  Stress 
Studies  for  presentation  at  the  ISTSS  2012  Annual  Meeting  in  Los  Angeles,  CA  (Nov.  1-3, 
2012). 
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APPENDIX  C:  TIMELINE 
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33 
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IRB  Protocols  Submission  and  Approval 
IRB  Protocol  Approval 
IRB  Protocol  Amendments 

Conduct  Focus  Groups  for  Adaptation  of  Web-based  CTI 

Plan  focus  group,  recruit  participants,  and  conduct  focus  group 
Transcribe  and  analyze  focus  group  data 
Modify  Web-based  Feedback  Message  to  Text  Messages 
Modify  Web-based  CTI  system  to  support  Text  Messaging 
Modify  language,  tone,  and  content  of  feedback  narratives  for  each  module 
Conduct  Beta  and  Usability  Testing  on  Cell  Phone 
Conduct  beta  testing  of  system 
Recruit  usability  testing  participants 
Conduct  usability  testing  with  veterans 
Modify  system  based  on  testing  data  as  needed 
Conduct  Pilot  Study 
TUX  System  Launch 
Particpant  recruitment 
Participant  consent  and  baseline  assessment 
Conduct  1 -month  online  assessment 
Conduct  3-month  post  study  assessment 
Prepare  Data 

Data  cleaning  and  normalization 
Data  transfer 

Conduct  analyses  to  test  Hypotheses 
Non-parametric  tests 
Analysis  of  variance 
Mediation  analysis 

Post  Intervention  Assessment  Meetings  (with  Pro-Change) 

Interpret  results 
Feasibility  Evaluation 
Submit  Final  Report 

Prepare  and  submit  final  report 

Prepare  draft  of  manuscript  for  publication 

Prepare  presentation  materials  for  scientific  meeting 


Duration 


Start 
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APPENDIX  D:  FOCUS  GROUP  FINAL  ANALYSIS  REPORT 


STRIVE  (CTI-PTSD):  Smoking  Focus  Group  Analysis 


Purpose  and  Methods 

Purpose:  This  report  summarizes  the  feedback  from  veterans  who  have  quit  smoking  that  participated 
in  the  focus  group  conducted  on  01/14/11  evaluating  the  smoking  expert  system  provided  by  ProChange 
Behavior  Systems,  Inc. 

Recruitment  &  Participants:  The  STR2IVE  study  team  recruited  individuals  with  flyers  and  word  of 
mouth.  Interested  participants  were  screened  and  if  they  qualified  they  were  informed  of  the  date  and 
time  of  the  focus  group.  Confirmation  calls  were  made  a  day  before  the  focus  group.  Ten  veterans  who 
have  quit  smoking  were  recruited  and  nine  participated  (one  no-show).  All  were  male. 

Procedure:  The  focus  group  methodology  was  informed  by  Morgan  (1998).  Two  practice  sessions  were 
conducted  to  familiarize  staff  with  the  procedures  and  to  finalize  the  protocol.  The  focus  group  began 
with  the  consenting  of  the  participants,  followed  by  introductory  comments  and  questions,  followed  by 
the  evaluation  of  the  expert  system  using  a  pre-established  focus  group  discussion  guide.  The  focus 
group  was  recorded  on  two  tape  recorders  and  a  cell  phone  placed  strategically  to  ensure  that  all 
discussion  points  were  captured.  The  focus  group  was  led  by  a  trained  moderator,  an  assistant 
moderator  was  present  to  take  notes  on  poster  paper  for  recap  of  points  during  the  focus  group,  and 
two  note  takers  were  present. 

The  focus  group  lasted  approximately  2  hours.  Food  and  water  were  provided  during  the  focus  group 
and  participants  received  a  $25  gift  card  at  the  end  of  the  focus  group  as  an  incentive  to  participate  and 
to  compensate  them  for  their  time.  Procedures  were  approved  by  VAPIHCS  IRB  and  UH  IRB. 

Analysis:  The  focus  group  analysis  followed  the  guidelines  recommended  by  Krueger  (1998). 

Immediately  after  the  focus  group,  the  moderator,  assistant  moderator,  and  note  takers  discussed  the 
group,  debriefing  and  noting  group  dynamics.  Prior  to  transcribing,  all  identifiers  were  removed  from 
the  materials  to  protect  confidentiality.  All  three  audio  recordings  were  used  to  develop  the  transcript, 
with  the  different  recordings  providing  better  sound  from  different  parts  of  the  room.  The  transcript 
was  then  compared  with  the  notes  taken  to  ensure  completeness.  Data  from  the  focus  groups  were 
coded  and  analyzed  according  to  published  protocols— which  summarize  the  major  themes  found  from 
the  groups  (Albright,  Maddock,  &  Nigg,  2004;  Lees,  Clark,  Nigg,  &  Newman,  2005;  Padula  et  al.,  2003; 

Pan  &  Nigg,  in  press).  This  entailed  breaking  the  transcriptions  into  meaning  units,  then  grouping  similar 
meaning  units  together  to  form  themes.  To  be  considered  a  theme,  an  idea  had  to  be  mentioned  at 
least  twice.  Common  themes  were  identified,  sorted,  and  compared. 

The  results  are  presented  by  categories  of  feedback  including  Opening  Questions:  Health  Information, 
and  then  addressing  the  expert  system  -  System,  Layout,  Text,  Graphics,  and  Audience  -  Issues  Specific 
to  Veterans.  Each  category  has  themes  which  are  comprised  of  specific  identified  meaning  units. 
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1.  Opening  Questions:  Health  Information 


Themes 

Number  of  Meaning  Units 

Benefits  of  quitting 

6 

Smoking  initiation 

2 

Support  for  quitting 

15 

Barriers  to  quitting 

18 

Benefits  of  quitting  (N=6):  Participants  mentioned  health  and  money,  social  acceptance,  personal 
hygiene,  prolonging  life  and  avoiding  property  damage.  Participants  also  discussed  only  being  able  to 
smoke  in  the  military  at  certain  times. 

Smoking  initiation  (N=2):  Three  participants  started  before  basic  training  and  one  started  after  basic 
training.  One  participant  started  smoking  after  leaving  the  Marine  Corps. 

Support  for  quitting  (N=15):  Participants  indicated  the  Veterans  Administration  (VA),  their  primary  care 
provider,  smoking  cessation  programs,  acupuncture,  hypnotism,  health  literature,  drugs  such  as  Chantix, 
and  nicotine  replacement  therapies  such  as  lozenges,  patches,  gum.  Participants  also  spoke  about 
superiors  during  active  duty  requiring  smoking  cessation. 

Barriers  for  quitting  (N=18):  Participants  listed  withdrawals,  stress,  weight  gain,  having  "nothing  to  do 
with  your  time,"  habit,  coffee  and  alcohol  consumption,  side  effects  from  medical  interventions,  failed 
attempts  at  cessation,  increased  autonomy  as  veterans,  and  visual  cues  that  promote  smoking  behavior. 

2.  System 


Themes 

Number  of 

Meaning  Units 

Slides  with  Meaning  Units  (in  descending  order) 

Clear 

14 

7  (N=5),  14  (N=5),  3A/B  (N=2),  13  (N=l),  18  (N=l) 

Unclear 

0 

Like 

10 

Lifestyle  (N=2),  LiveWell  (N=8) 

Dislike 

0 

Neutral 

1 

2A/B (N=l) 

Suggestion 

3 

Closing  (N=3) 

Clear  (N=14):  Participants  found  that  the  LiveWell  format  was  straightforward.  The  Stages  of  Change  on 
slide  7  was  also  understandable.  Participants  were  able  to  answer  the  questions  on  slide  13  and  fill  out 
the  blank  on  slide  14  correctly.  On  slide  18  they  indicated  they  would  be  able  to  set  a  goal. 

Like  (N=10):  Participants  stated  that  the  Lifestyle  system  "grabs  your  attention."  However,  the  LiveWell 
system  garnered  greater  positive  feedback.  Participants  liked  its  overall  aesthetic,  clarity,  the  stages  of 
change  and  implied  credibility  ("30  years  of  research"). 
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Neutral  (N=l):  One  participant  had  a  neutral  opinion  towards  the  comparison  of  the  two  systems. 

Suggestion  (N=3):  In  the  closing  questions  participants  discussed  friendly  vs.  scientific  language.  One 
stated  that  it  should  be  a  combination  of  both,  another  stated  scientific  if  explanations  were  provided 
and  a  third  agreed. 


SUMMARY:  The  LiveWell  system  seems  to  be  the  preferred 
system  and  the  system  itself  is  clear  and  understandable. 


3.  Layout 


Themes 

Number  of 

Meaning  Units 

Slides  with  Meaning  Units  (in  descending  order) 

Clear 

0 

Unclear 

1 

15  (N=l) 

Like 

1 

3B  (N=l) 

Dislike 

0 

Neutral 

1 

Closing  (N=l) 

Suggestion 

7 

Closing  (N=6),  15  (N=l) 

Unclear  (N=l):  One  participant  was  unclear  about  the  status  bar. 

Like  (N=l):  One  participant  preferred  the  layout  of  the  LiveWell  system  because  it  focuses  your 
attention  on  the  words. 

Neutral  (N=l):  One  participant  articulated  that  they  had  no  opinion  about  the  layout  color  scheme. 

Suggestion  (N=7):  One  participant  thought  the  Lifestyle  system  should  use  green,  use  more  contrasting 
colors  and  update  the  pictures.  Two  participants  suggested  a  pink  color  scheme,  two  suggested  green 
and  another  suggested  blue. 


SUMMARY:  The  main  comments  about  the  layout 
addressed  color  with  green  and  pink  preferred. 


4.  Text 


Themes 

Number  of 

Meaning  Units 

Slides  with  Meaning  Units  (in  descending  order) 

Clear 

36 

10  (N=9),  6  (N=6),  9  (N=4),  8  (N=4),  12  (N=3),  14 

3 


(N=3),  7  (N=3),  11  (N=l),  3A/B  (N=l),  15  (N=l),  5 
(N=l) 

Unclear 

24 

6  (N=ll),  9  (N=5),  5  (N=4),  8  (N=3),  3A/B  (N=l) 

Like 

26 

4  (N=7),  2A/B  (N=4),  8  (N=3),  13  (N=3),  10  (N=3),  18 
(N=2),  16  (N=2),  14  (N=l),  3A/B  (N=l) 

Dislike 

5 

10  (N=2),  3A/B  (N=l),  12  (N=l),  8  (N=l) 

Neutral 

0 

Suggestion 

11 

Closing  (N=3),  9  (N=2),  6  (N=l),  4  (N=l),  11  (N=l),  5 
(N=l),  14  (N=l),  12  (N=l) 

Clear  (N=36):  On  slide  2B,  three  participants  felt  that  the  LiveWell  system  was  more  direct.  On  slide  5, 
the  text  was  clear,  on  slide  6  the  question  was  clear  and  the  answer  choices  were  clear  if  you  had  quit 
smoking  or  if  you  read  through  all  of  them  first.  On  slide  7,  participants  were  able  to  explain  the  stages 
of  change.  On  slide  8  they  were  able  to  explain  the  meaning  of  the  text  in  the  paragraph  and  the  bullet 
points,  which  they  also  found  thought-provoking.  On  slide  9,  participants  were  able  to  correctly  explain 
the  instructions  in  their  own  words.  Participants  agreed  that  the  questionnaire  on  slide  10  was  asking 
you  to  rate  the  importance  of  each  item,  not  whether  or  not  you  agreed  with  it.  On  slide  11, 
participants  understood  the  bullet  points.  Participants'  comments  about  slides  12,  14  and  15  also 
demonstrated  understanding. 

Unclear  (N=24):  On  slides  3A/B,  one  participant  commented  that  "contemplation"  might  not  be  a  word 
most  people  know.  On  slide  5,  several  participants  were  confused  by  "transtheoretical"  and  one 
suggested  keeping  the  text  at  a  5th  grade  reading  level.  On  slide  6  several  participants  found  the 
question  misleading  and  said  they  would  choose  more  than  one  answer.  On  slide  8,  participants  were 
not  sure  how  someone  could  practice  to  quit.  On  slide  9  participants  found  the  instructions  confusing 
because  they  assume  that  the  user  knows  smoking  constitutes  a  health  risk  and  that  that  is  important. 

Like  (N=26):  On  slide  2A,  participants  liked  "at  my  pace,"  and  on  slide  2B,  participants  liked  the  stages  of 
change.  On  slide  3A  one  participant  liked  the  scientific  aspect  of  the  text.  On  slide  4,  participants  liked 
the  aspect  of  personal  choice  implied  in  "at  your  own  pace."  On  slide  8  one  participant  liked  the  24- 
hour  trial  suggestion  and  another  liked  the  suggestion  of  nicotine  replacement  therapies.  On  slide  10 
participants  found  the  text  "eye-opening"  and  relatable.  On  slide  13,  participants  found  the  hints  and 
strategies  helpful.  On  slide  14  one  participant  appreciated  the  encouraging  tone  of  the  text.  On  slide 
16,  participants  liked  the  suggestion  of  asking  friends  and  family  for  support.  On  slide  18,  one 
participant  liked  the  information  about  cutting  back  on  how  many  cigarettes  you  smoke  and  another 
liked  the  prompting  to  decide  when  to  quit. 

Dislike  (N=5):  On  slide  3A/B,  one  participant  felt  the  text  was  "preaching"  at  him.  On  slide  8  one 
participant  said  that  the  number  of  cigarettes  you  smoke  per  day  is  not  as  important  as  how  much  you 
enjoy  smoking.  On  slide  10  one  participant  did  not  agree  with  "Quitting  can  improve  my  appearance" 
and  "Quitting  can  make  it  hard  to  be  around  other  smokers"  because  he  would  quit  for  himself,  not 
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other  people.  On  slide  12,  one  participant  did  not  find  "I'm  not  hurting  anyone  else"  believable  because 
everyone  has  family. 

Suggestion  (N=ll):  One  participant  thought  that  "at  your  own  pace"  was  risky  because  if  it  were  at  his 
pace  it  would  never  happen.  Another  participant  felt  strongly  that  the  directions  should  be  clearer  to 
choose  one  answer  on  slide  6.  On  slide  9,  participants  felt  that  the  answer  choices  "very"  and 
"extremely"  were  too  similar  and  that,  overall,  the  slide  could  be  simplified.  On  slide  11,  one  participant 
felt  that  the  examples  were  relatable  but  not  the  most  important  ones.  On  slide  12,  one  participant  said 
they  would  like  to  see  more  selection  for  example,  instead  of  "I'm  not  old  enough"  something  more 
along  the  lines  of,  "I'm  too  old  and  it  hasn't  killed  me  yet  so  hell  with  it  it's  too  late  to  quit  now."  On 
slide  14,  one  participant  suggested  putting  non-smoking  activities  like  swimming  and  running.  In  the 
closing  questions,  participants  recommended  using  more  facts  and  scientific  evidence  in  the  text  like 
mortality  rates. 

SUMMARY:  The  majority  liked  the  text,  they  picked  up  that  it  is 
not  forcing  you  to  quit  and  the  strategies  and  examples  used 
seem  to  fit  well.  The  main  challenge  identified  with  the  text  was 
that:  a)  the  stage  question  needs  to  be  clarified  so  that  it  is  clear 
that  it  means  quitting  for  good  and  that  only  one  answer  should 
be  chosen;  and  b)  possibly  adding  more  examples  and  strategies. 


Graphics 


Themes 

Number  of 

Meaning  Units 

Meaning  Units  (in  descending  order) 

Clear 

20 

9  (N=5),  14  (N=4),  6  (N=3),  7  (N=2),  4  (N=2),  11 
(N=l),  5  (N=l),  16  (N=l),  13  (N=l) 

Unclear 

24 

18  (N=5),  4  (N=4),  8  (N=4),  5  (N=3),  6  (N=3),  14 
(N=2),  11  (N=l),  12  (N=l),  15  (N=l) 

Like 

16 

2A/B  (N=6),  13  (N=4),  9  (N=3),  11  (N=l),  16  (N=l), 
4  (N=l) 

Dislike 

24 

6  (N=5),  5  (N=5),  16  (N=4),  8  (N=4),  11  (N=3),  13 
(N=l),  14  (N=l),  4  (N=l) 

Neutral 

7 

11  (N=3),  12  (N=2),  6  (N=l),  18  (N=l) 

Suggestion 

15 

Closing  (N=3),  7  (N=2),  15  (N=2),  4  (N=2),  5  (N=2), 
11  (N=l),  8  (N=l),  6  (N=l),  18  (N=l) 

Clear  (N=20):  The  birds  on  slide  2A  indicated  fresh  air  and  freedom.  The  picture  on  slide  5  was  identified 
as  an  EKG  to  track  your  heart.  The  burning  cigarettes  on  slide  6  were  associated  with  the  stages  of 
change.  The  image  on  slide  7  demonstrated  the  user's  stage  of  change.  The  sunset  on  slide  9  suggested 
a  new  beginning,  a  journey  and  a  destination.  The  couple  on  slide  11  looked  healthy  with  good  hygiene. 
The  father  and  son  on  slide  13  represented  a  healthy  lifestyle.  The  couple  on  slide  14  showed  a  father 


5 


and  daughter  supporting  each  other  and  perhaps  an  older  man  who  had  quit  smoking  to  promote  life. 
The  image  on  slide  16  showed  confidence. 


Unclear  (N=24):  The  birds  on  slide  4  seemed  off  topic.  The  image  on  slide  5  was  difficult  to  identify  as  a 
cigarette.  Participants  found  the  image  on  slide  6  confusing.  The  image  on  slide  8  came  across  as  a 
napkin,  not  a  pack  of  cigarettes.  The  image  of  the  man  on  slide  12  conveyed  disappointment.  The 
woman  on  slide  14  looked  like  a  "gold  digger"  or  a  "mistress."  The  background  of  the  image  on  slide  15 
was  unclear  and  could  be  a  jungle  gym,  a  cafe  or  a  snack  bar.  The  text  in  the  image  on  slide  18  was  not 
legible. 

Like  (N=16):  Participants  liked  the  birds  on  slides  2A  and  4,  the  shadows  on  slide  2B,  the  sunset  on  slide 
9,  the  couple  on  slide  11,  the  father  and  son  on  slide  13  and  the  fist  on  slide  16. 

Dislike  (N=24):  Participants  did  not  like  the  birds  on  slides  2A  and  4,  the  EKG  on  slide  5,  the  burning 
cigarettes  on  slide  6,  the  crushed  cigarettes  on  slide  8,  the  couple  on  slide  11,  the  father  and  son  on  slide 
13,  the  couple  on  slide  14  and  the  fist  on  slide  16. 

Neutral  (N=7):  Participants  felt  neutral  about  the  couple  on  slide  11,  the  man  on  slide  12,  the  burning 
cigarettes  on  slide  6  and  the  blackboard  on  slide  18. 

Suggestion  (N=15):  Participants  suggested  a  tombstone  or  cemetery  on  slide  4,  showing  the  filter  of  the 
cigarette  on  slide  5  and  showing  smokers'  lungs  or  students  in  cessation  classes  on  slide  7.  They  also 
recommended  making  the  box  look  more  realistic  on  slide  8,  showing  a  person  looking  in  the  mirror  on 
slide  11,  not  using  images  that  trigger  smoking  behavior  like  the  background  on  slide  15,  making  the 
pictures  look  more  local  to  Hawaii  and  making  the  writing  on  slide  18  legible.  In  the  closing  questions, 
participants  suggested  more  pictures  that  depict  the  impact  of  smoking  on  your  body. 


SUMMARY:  Although  most  participants  understood  why  the 
graphics/pictures  were  included,  the  majority  did  not  like  the 
graphics/pictures  because  they  were  ambiguous  to  some  extent, 
which  allowed  for  negative  interpretations.  The  participants  did 
provide  some  concrete  suggestions  as  to  what  pictures  or  aspects 
could  be  changed. 


6.  Audience  -  Issues  Specific  to  Veterans 


Themes 

Number  of 

Meaning  Units 

Slides  with  Meaning  Units  (in  descending  order) 

Clear 

0 

Unclear 

0 

6 


Like 

10 

4  (N=7),  8  (N=3) 

Dislike 

4 

16 (N=4) 

Neutral 

0 

Suggestion 

2 

15 (N=2) 

Like  (N=10):  On  slide  4,  participants  noted  that  they  liked  the  mention  of  "at  your  own  pace"  because  it 
promotes  autonomy  which,  one  participant  stated,  is  why  they  left  the  military  and  another  said  is 
relatable  for  veterans  who  appreciate  having  greater  control  over  their  life.  On  slide  8,  participants 
agreed  that  this  slide  was  appropriate  for  veterans. 

Dislike  (N=4):  Participants  did  not  think  that  the  fist  on  slide  16  was  realistic  for  veterans,  who  would  be 
more  likely  to  form  a  fist  with  their  thumb  on  the  outside  of  their  fingers. 

Suggestion  (N=2):  One  participant  thought  that  the  slides  should  be  "localized"  to  reflect  veterans' 
lifestyle  in  Hawaii. 


SUMMARY:  In  general  the  participants  felt  that 
this  system  was  appropriate  for  Veterans. 
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APPENDICES 


APPENDIX  E:  USABILITY  TESTING  SUMMARY  REPORT 


Pl-Ax  only 


Pl-Ax  only 


2/27 


Time 

Type 

Pos/Neg 

User  Event 

User  Event  Description 

Screen  Description 

0:03:15 

Observation 

Structure  -  layout 

Participant  said  she  wasn't  sure  what  to  do  here.  Said  "1  don't  know... 
register?"  and  clicked  on  Register  without  assistance. 

Log-In 

0:05:23 

Observation 

Participant  said  she  doesn't  like  choosing  log-ins,  passwords,  security 
questions  because  she  has  a  bad  memory  and  would  have  difficulty 
remembering  them.  Said  this  would  stress  her  out  but  did  not  have  any 
suggestions  for  alternatives. 

Register 

0:07:23 

Observation 

Content  -  language 
(instructions) 

Participant  asked  if  the  question  was  asking  specifically  about  "combat" 
experience  or  any  truama  while  in  the  military.  Said  she  would  answer 
"yes"  for  the  former  and  "no"  for  the  latter.  Thought  it  should  say,  for 
example,  "combat  or  non-combat  experience". 

Screening  Questions 

0:09:49 

Px  prompted 

Reminded  participant  to  pretend  the  observers  weren't  there  and  that  if 
the  participant  wouldn't  normally  read  through  the  Study  Fact  Sheet 
then  she  should't  feel  obligated  to.  So  participant  went  straight  to 
"agree". 

Study  Fact  Sheet 

0:13:01 

Px  prompted 

Content  -  language 

(questionnaire 

items) 

Asked  what  the  participant  thought  of  the  questions.  Participant  said 
that  answering  some  of  the  questions  were  difficult  because  she  really 
has  to  think  about  her  feelings  or  think  about  the  best  fitting  response 
based  on  the  available  options.  She  said  some  just  have  more 
complicated  answers  and  has  nothing  to  do  with  the  system/program 
itself. 

Military  Experiences 

0:17:03 

Px  prompted 

Content  -  language 
(response  options) 

Asked  participant  what  she  was  currently  thinking.  Participant  said  the 
reponse  options  could  be  improved.  She  suggested  "not  at  all,  rarely, 
sometimes,  all  the  time"  would  make  more  sense  to  her  than  "not  at  all, 
several  days,  more  than  half  the  days,  nearly  every  day."  Said  the  current 
response  options  would  probably  be  confusing  to  veterans. 

General  Questions 
(Depression  ) 

0:26:40 

Observation 

Content  -  options 

Participant  said  she  didn't  know  which  ONE  race  to  choose  since  she  is 
"half  and  half."  Said  it  is  "irritating"  and  it  should  be  changed  to  "all  that 
apply." 

Demographic 

Questions 

CTI  Usability  Testing  Summaries  by  Participant 
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Pl-Ax  only 


Pl-Ax  only 


2/27 


Time 

Type 

Pos/Neg 

User  Event 

User  Event  Description 

Screen  Description 

0:29:32 

Px  prompted 

Content  -  language 
(response  options) 

Asked  participant  what  she  thought  of  this  screen.  Said  that  she  didn't 
deploy  so  she  thought  there  should  be  an  "n/a"  or  "none"  option.  Didn't 
enter  anything  and  went  to  Next  button. 

Military  Background 

0:30:15 

Observation 

Content  -  language 

(questionnaire 

items) 

Participant  wanted  to  know  what  was  considered  "dangerous  duty"  since 
she  wasn't  in  combat,  in  order  to  respond  to  this  item. 

Combat  Experience 

0:31:49 

Px  prompted 

Structure  - 
layout/text. 

Asked  participant  why  she  moved  forward  in  her  chair  at  this  point. 
Participant  said  it  was  because  she  couldn't  tell  if  the  response  option 
said  "1  dash  3X"  or  "13X".  When  prompted,  respondedthat  she  easily 
understoon  that  "X"  meant  times. 

Combat  Experience 

0:33:55 

Observation 

Content  -  language 

(questionnaire 

items) 

Participant  said  "rounds"  is  too  specific  since  lots  of  other  things  can 
happen  to  veterans  e.g.,  something  could  fall  on  them,  be  in  an 
explosion, e  tc.  And  it  doesn't  acurately  reflect  the  experiences  of 
veterans.  Seems  like  she  felt  insulted  that  it  didn't  accurately  capture 
other  traumatic  experiences. 

Combat  Experience 

0:40:48 

Error 

Error 

Participant  pressed  the  Next  button  before  answering  any  of  the  items 
on  the  screen  and  got  an  error  message.  When  prompted,  P  said  she 
hadn't  noticed  what  she  did  or  the  error  message.  Said  she  noticed  the 
"red"  but  not  the  message  itse.f. 

Perceived  Stress 

0:44:28 

Px  prompted 

Content  -  language 
(instructions) 

Asked  participant  what  she  thought  about  the  instructions.  Participant 
said  it  was  confusing  and  she  had  to  read  it  3  times.  Said  she  feels  better 
after  having  re-read  it  and  thinks  it  could  be  reworded.  Then  she  read  it 
several  more  times. 

Quality  of  Life 

0:49:14 

Observation 

Content  -  language 
(response  options) 

Participant  asked  "what's  the  difference  between  delighted  and 
pleased?"  regarding  the  response  options.  Said  she  would  change  it  to 
"very  pleased"  and  "pleased"  or  "0  to  5"  to  make  it  more  clear.  (Note:  it 
took  her  3  minutes  to  answer  the  1st  question.) 

Quality  of  Life 

CTI  Usability  Testing  Summaries  by  Participant 
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Pl-Ax  only 


Pl-Ax  only 


2/27 


Time 

Type 

Pos/Neg 

User  Event 

User  Event  Description 

Screen  Description 

0:53:06 

Other - 

interviewer 

note 

Content  -  language 

(questionnaire 

items) 

Participant  thought  there  should  be  an  "n/a"  option  for  "having  and 
rearing  children".  Didn't  connect  the  instructions  to  this  item  even 
though  she  read  it  10+  times.  She  got  frustrated  and  selected  a  random 
response  to  move  on. 

Quality  of  Life 

0:56:40 

Observation 

Content  -  language 

Participant  thought  the  exercise  definition  might  be  too  extreme 
(inaccurate).  Said  she  only  does  light  swimming  twice  a  week  which 
might  add  up  to  2  hours  and  30  minutes. 

Exercise 

0:57:59 

Px  prompted 

Content  -  language 

Asked  participant  what  she  thought  the  purpose  of  the  exercise  defition 
screen  was  and  participant  said  she  had  no  idea.  Participant  said  she 
thought  it  might  be  trying  to  make  people  feel  bad  if  they  weren't 
exercising  up  to  "regular  exercise"  defitions. 

Exercise 
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2/27 


P2-SC 


R2P1-SC-R1P1-SM-DP-052511 


Time 

Type 

Pos/Neg 

User  Event 

User  Event  Description 

Screen  Description 

0:02:24 

Other - 

interviewer 

note 

Structure  -  layout 

Participant  recognized  that  he  was  a  New  User  immediately  after  being 
reminded  to  imagine  he  was  doing  this  program  at  home  alone.  No 
difficulties  going  to  the  Register  button  (old  login  screen). 

Log-In 

0:04:09 

Other - 

interviewer 

note 

Technical 

Participant  enlarged  the  screen  on  his  own. 

Screening  Questions 

0:10:04 

Observation 

Content  -  language 

Wasn't  sure  how  to  answer  some  questions  as  a  former  sailor.  Thought 
some  questions  were  geared  toward  soldiers  in  direct  combat  situations. 
He  had  similar  experiences  but  not  exactly  like  the  questions. 

Combat  Experience 

0:11:00 

Observation 

Content  -  language 

Similar  to  above  issue:  he  had  difficulty  responding  to  item  "How  often 
did  you  fire  rounds  at  the  enemy?"  since  he  said  he  fired  "warning  shots." 
Felt  it  should  be  considered  more  than  "never"  but  didn't  feel  like  he  was 
answering  the  question  based  on  the  wording  so  he  ended  up  selecting 
"never." 

Combat  Experience 

0:14:55 

Other - 

interviewer 

note 

Content  -  language 

Participant  quickly  understood  that  the  item  about  satisfaction  with 
"having  and  rearing  children"  also  included  satisfaction  with  NOT  having 
them  and  entered  "delighted." 

Quality  of  Life 

0:17:15 

Observation 

Structure  -  layout; 
Content  -  language 

Said  "there's  no  answer  box..."  meaning  that  he  was  expecting  to  answer 
the  Exercise  definiton  screen.  Went  to  the  next  screen  anyway  and 
answered  the  question  there. 

Exercise 

0:22:37 

Px  needed 
help 

Structure  - 
layout/text; 

Content  -  language 

Looked  for  "next"  button  at  the  bottom  of  the  screen  so  he  wasn't  sure 
what  to  do  when  there  wasn't  one.  Thought  that  what  he  already  did 
(HRI)  was  the  Smoking  Cessation  program  even  though  he  read  the 
instructions  out  loud  that  he  was  supposed  to  go  to  the  programs  to  the 
left  and  that  the  numbers  indicated  how  many  times  those  programs 
were  completed.  Ended  up  to  gift  card  information  instead. 

Home  Page 
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P2-SC 


R2P1-SC-R1P1-SM-DP-052511 


2/27 


Time 

Type 

Pos/Neg 

User  Event 

User  Event  Description 

Screen  Description 

0:38:31 

Observation 

Content  -  language 

Wasn't  clear  what  the  question  was  asking  regarding  how  many 
cigarettes  he  was  willing  to  decrease  each  day.  Thought  it  might  mean  if 
he  selected  "2  cigarettes"  it  would  mean  on  day  2  he  would  have  to 
decrease  it  by  4,  etc. 

Smoking  Cessation  - 
Small  Steps 

0:40:05 

Other  - 

interviewer 

note 

Participant  wasn't  sure  what  to  do  next  -  probably  since  he  thought  we 
had  an  agenda  for  what  he  should  work  on  during  the  usability  testing. 
When  asked,  he  said  that  he  probably  wouldn't  have  gone  to  the  Report 
if  he  was  home  alone,  but  went  to  it  anyway  "because  it's  first"  (before 
the  Continue  button). 

Thank  You 

0:42:08 

Other - 

interviewer 

note 

Participant  didn't  have  any  specific  comments  regarding  the  Report. 

Report 

0:42:24 

Px  prompted 

Structure  - 
images/text 

When  asked,  participant  said  he  was  primarily  paying  attention  to  the 
text  and  not  the  images  so  none  of  them  stood  out  to  him. 

At  Thank  You 

0:43:56 

Px  prompted 

General 

Asked  participant  if  he  thought  this  program  could  help  him  and  he  said 
"Yeah !  Actually  I'm  stoked..."  and  asked  if  he  could  come  back  to  use  the 
program.  Informed  him  that  the  feasibility  study  wouldn't  be  up  and 
running  for  a  few  weeks  but  he  could  participate  in  the  TUX  study  now. 

At  Thank  You 
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P2-SM-DP 


R2P1-SC-R1P1-SM-DP-052511 


Time 

Type 

Pos/Neg 

User  Event 

User  Event  Description 

Screen  Description 

0:13:14 

Observation 

Content  -  spelling 

error 

Noticed  a  spelling  error.  Says  "fried"  instead  of  "friend." 

Your  Level  of 
Confidence  (SM) 

0:21:21 

Observation 

Content  -  language 

Said  "it's  encouraging!"  regarding  increasing  stress  management  goals 
(increasing  time  spent  managing  your  stress  in  the  next  month) 

Small  Steps 

0:21:47 

Observation 

Content  -  language 

Same  issue  as  in  SC  program  -  unsure  what  the  question  is  asking  him 
regarding  how  many  minutes  he  is  willing  to  increase  each  day  managing 
stress.  (He  put  the  lowest  option  (i.e.,  5  min)  because  he's  interpretting 
it  as  he  has  to  increase  stress  management  by  10  min  on  day  2,  15  min 
on  day  3,  etc.) 

Small  Steps 

0:22:52 

Px  prompted 

Content  -  language 

Asked  what  P  thought  of  the  SM  recommendations.  He  said  it  was  good. 

At  Thank  You 

0:23:02 

Px  prompted 

Content  -  language 

Laurel  asked  about  adding  minutes  of  SM  each  day.  Participant  said  it 
sounded  like  he  had  to  keep  increasing  the  number  of  minutes  by  that 
number  each  day  and  joked  that  by  October  he'd  have  to  quit  his  job 
because  he's  be  managing  his  stress  all  day. 

At  Thank  You 

0:38:22 

Other  - 

interviewer 

note 

Content  -  language; 
Structure  -  text  box 

P  had  a  little  difficulty  with  the  fill-in-the  blank  that  asked  him  to  "type 
one  thing  you've  been  doing  to  prevent  depression  that  deserves  credit. 
Then  type  in  how  you  can  reward  yourself"  and  commented  that  there's 
only  one  box  so  he  had  to  re-read  the  question  a  few  times. 

Your  Own 

Experiences  - 

Reward  Yourself 

0:42:20 

Px  prompted 

Content  -  language 

Asked  P  if  he  thought  the  DP  program  would  help  someone  at  his  SoC 
(Maintenance).  He  said  "yes"  because  he  thought  it  was  interactive 
without  being  judgmental  or  telling  him  what  to  do  like  a  counselor 
might.  He  said,  as  a  veteran/out  of  the  military,  he  doesn't  want  to  be 
told  what  to  do. 

At  Thank  You 
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2/27 


P3-SC 


R2P3-SC-052611 


Time 

Type 

Pos/Neg 

User  Event 

User  Event  Description 

Screen  Description 

0:02:24 

Observation 

Wasn't  sure  if  he  should  register  or  log-in.  Took  him  about  40  seconds  of 
talking  through  it  until  he  went  to  Register  without  help. 

Log-In 

0:03:17 

Other  - 

interviewer 

note 

Content  -  language 

Participant  asked  if  there  was  a  "demo"  for  assistance  in  selecting  a  log¬ 
in,  passowrd,  etc.  (Note:  maybe  could  provide  parameters  e.g.,  not  case- 
sensitive,  must  be  at  least  four  characters,  etc.) 

Register 

0:05:07 

Observation 

Technical 

Commented  that  he  had  to  scroll  down  to  see  the  last  security  question. 

Register 

0:05:56 

Error 

Error 

Read  the  1st  screening  question  as  "are  you  in  the  military"  instead  of 
"military  veteran"  so  he  selected  "no".  Asked  him  how  he  read  that 
question  and  then  he  saw  his  error.  Would  have  screened  out  for  this. 

Screening  Questions 

0:08:10 

Observation 

Wasn't  sure  if  he  was  answering  each  of  the  four  PTSD  criteria  separately 
or  together  even  though  there  was  only  one  "yes/no"  option.  He  said  he 
would  say  "yes"  to  2  items  and  "no"  to  the  other  2  items. 

Screening  Questions 

0:09:01 

Other  - 

interviewer 

note 

Technical 

Experiencing  difficulty  with  Internet  -  slow  to  respond  and  "Internet 
Explorer  cannot  display  the  webpage." 

At  Screening 
Questions 

0:10:16 

Other  - 

interviewer 

note 

Answered  "yes"  to  SMI  screening  question.  Said  he  didn't  know  what 
Bipolar  Disorder.  Not  sure  what  he  was  endorsing  but  he  would've 
screened  out  here  as  well. 

Screening  Questions 

0:12:59 

Other  - 

interviewer 

note 

Structure  -  format; 
Content  -  language 

Looked  for  way  to  "answer"  the  exercise  definition  screen.  Decided  to 
click  next  without  assistance. 

Exercise 

0:17:42 

Other - 

interviewer 

note 

Technical 

Realized  that  we  told  Pro-Change  the  appointment  was  cancelled  and 
forgot  to  tell  them  when  we  filled  the  openning  at  the  last  minute.  Might 
be  the  cause  of  the  technical  difficulties  if  Pro-Change  was  working  on 
the  system  at  the  same  time. 

At  Healthy  Eating 
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P3-SC 


R2P3-SC-052611 


2/27 


Time 

Type 

Pos/Neg 

User  Event 

User  Event  Description 

Screen  Description 

0:35:42 

Other - 

interviewer 

note 

Structure  -  images; 

Technical 

Images  did  not  show  up  on  the  majority  of  the  screens  due  to  technical 
issues  so  participant  didn't/couldn't  comment  on  them. 

General 
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P4-SM 


R1P2-SM-060111 


2/27 


Time 

Type 

Pos/Neg 

User  Event 

User  Event  Description 

Screen  Description 

0:01:39 

Observation 

Negative 

Structure  -  Color 

Doesn't  like  the  colors 

Log-In 

0:02:32 

Error  -  resolved 
by  participant 

Structure  -  Layout 

Logged  in  to  Returning  User  then  got  error  message  saying 
previously  not  registered.  So  went  to  New  User  to  register 
without  assistance. 

Log-In 

0:03:30 

Observation 

Content?  - 
Unexpected 

Security  questions  are  different  from  those  normally  asked 

Security  Questions 

0:04:51 

Observation 

Structure  -  Text 

1  would  just  skim  this 

Study  Fact  Sheet 

0:06:14 

Observation 

Positive 

Content?  - 

Informative 

Likes  that  Fact  Sheet  provides  contact  info  if  participant  needs 
help 

Study  Fact  Sheet 

0:11:17 

Observation 

Content  -  Instructions 

Confused  about  instructions  for  satisfaction  but  continues 

Quality  of  Life 
Questionnaire 

0:15:32 

Observation 

Structure  -  Layout 

Wishes  the  screen/font  was  bigger  (not  used  to  laptop-size). 
Told  her  she  could  maximize  the  screen. 

Healthy  Eating 

0:17:07 

Observation 

Negative 

Content  -  Language 

Prefer  if  these  (types  of)  questions  were  asked  in  a  different 
way.  Thinks  they  "put  [her]  on  the  spot"  or  are  "finger 
pointing." 

Healthy  Eating  & 
Responsible 

Drinking 

0:18:56 

Observation 

Negative 

Structure  -  Text 

A  lot  of  words  on  the  screen  which  may  be  difficult  for  older 
Veterans  or  those  with  severe  depression/PTSD.  Said  she  likes 
to  read  &  is  in  school  now  &  she  thinks  the  screens  are 
"wordy"  &  she  feels  "edgy." 

Preventing 

Depression 

0:20:00 

Observation 

Content  -  Response 
Options 

Thinks  response  options  are  too  narrow.  But  then  says  maybe 
we  wanted  to  categorize  responses  in  this  way. 

Preventing 

Depression 

0:20:43 

Observation 

Structure  -  Image 

Not  sure  what  image  is  (next  to  the  word  "wellness"). 

Thank  You 

0:24:44 

Observation 

Positive 

Structure  -  Image 

Liked  the  picture  of  person  sitting  on  a  lounge  chair;  wished  it 
was  bigger;  already  makes  her  feel  calm. 

SM  Program 
Homepage 

0:28:59 

Error  -  resolved 
by  participant 

Missed  an  item/radio  button  &  got  an  error  message.  Read 
the  message,  answered  the  item  &  moved  on. 

SM  Activities 

Questionnaire 

0:30:43 

Observation 

Positive 

Structure  -  Image 

Likes  pictures  with  people  in  them 

Activities  & 
Strategies:  Taking  a 
Healthier  Approach 
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P4-SM 


R1P2-SM-060111 


2/27 


Time 

Type 

Pos/Neg 

User  Event 

User  Event  Description 

Screen  Description 

0:30:48 

Px  Prompted 

Negative 

Content  -  Feedback 

Px  was  asked  what  she  thought  of  the  feedback  since  she 
hadn't  made  any  comments  on  the  last  few  screens.  Reported 
that  they  seemed  too  "general."  She  suggested  using 
examples  but  was  unable  to  provide  specific  suggestions. 

Activities  & 
Strategies:  Taking  a 
Healthier  Approach 

0:31:29 

Observation 

Positive 

Structure  -  Image 

Said  she  "definitely"  likes  the  picture  of  the  ladybug  on  the 
yellow  flower. 

Pros  &  Cons 

0:32:34 

User  needs  help 

Negative 

Content  -  Instructions 

Confused  about  how  to  answer  the  questions.  Siad  if  she  were 
by  herself,  she  would  just  "muddle  through  it." 

Pros  &  Cons 

Questionnaire 

0:34:26 

User  needs  help 

Negative 

Content  -  Instructions 

Doesn't  understand  what  the  screen  means  by  "what  is  your 
most  important  Pro  for  managing  stress?" 

Weigh  the  Pros  & 

Cons 

0:35:46 

Observation 

Negative 

Content  -  Language 

Thinks  feedback  is  poorly  worded.  Said  she  would  "laugh  at 
that." 

Your  Level  of 

Confidence 

0:36:31 

Observation 

Negative 

Content  -  Language 

Thinks  feedback  is  inappropriate  based  on  her  responses  to  the 
questionnaire  items.  Thinks  asking  someone  who  IS  confident 
what  they  do  (feedback  provided  by  program)  is  not  the  first 
thing  she  would  do  &  that  she  wouldn't  even  be  able  to 
recognize  if  someone  was  confident  in  order  to  ask  them. 

Your  Level  of 

Confidence 

0:37:17 

Observation 

Negative 

Content  -  Language 

Thinks  feedback  could  be  worded  differently  to  be  "more 
empathetic"  and  less  "you,  you  ,  you."  But  she  liked  the 
suggestions. 

Your  Level  of 

Confidence 

feedback 

0:38:13 

Observation 

Positive 

Content  -  Instructions 

Said  she  likes  to  answer  these  types  of  questionnaires  because 
she  understands  what  they're  asking  of  her  and  she  can 
answer  them  quickly  (e.g.,  as  opposed  to  the  other 
questionnaires  that  ask  how  important  was  blank  in  your 
decision  to...) 

Your  Own 

Experience 

Questionnaire 

0:41:00 

Observation 

Negative 

Content  -  Language 

Said  that  feedback  could  be  elaborated  on  with  more  specific 
examples  (similar  to  her  other  comment  on  the  feedback 
screens). 

Your  Strategies  for 
Change  feedback 
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P4-SM 


R1P2-SM-060111 


2/27 


Time 

Type 

Pos/Neg 

User  Event 

User  Event  Description 

Screen  Description 

0:41:38 

Observation 

Negative 

Content  -  Language 

Again,  participant  thinks  feedback  could  be  reworded,  using 
less  "you"  and  "your"  type  feedback  so  it  sounds  less  blaming. 

Your  Strategies  for 
Change  feedback 

0:42:33 

Observation 

Positive 

Content  -  Language 

Likes  the  exercise  of  writing  things  down  (3  fill-in-the-blank 
boxes)  because  it  reminds  her  of  exercises  that  her  e.g., 

Your  Strategies  for 
Change:  Use 

0:43:15 

Observation 

Content  -  Language 

Said  there  was  not  enough  elaboration  on  feedback  but  it  was 
still  overall  because  she  likes  using  calendars  to  remind  her. 

Your  Strategies  for 
Change:  Make  a 

Commitment 

0:43:59 

Observation 

Positive 

Content  -  Language 

Likes  that  it  gives  websites.  Said  she  would  go  to  the  site  to  get 
ideas. 

Your  Strategies  for 
Change:  Get  Support 

0:45:19 

Observation 

Positive 

Content  -  Language 

Likes  when  the  program  gives  positive 
feedback/encouragement,  for  ex.,  "Way  to  go!"  Thinks  it 
would  make  most  people  feel  good. 

Small  Steps 

0:46:43 

Observation 

Content  -  Language 

Said  she  would  read  the  whole  report  and  when  asked,  said 
she  thinks  she  would  print  it  out  as  well. 

Report 

0:48:48 

Observation 

Content  -  Language 

Said  she  doesn't  use  Amazon  much,  but  getting  a  gift  card  is 

nice. 

Report 

0:50:14 

Observation 

Positive 

Content  -  Language 

Likes  that  it's  tailored  (SoC)  to  where  the  person  is  in  their 
management  of  stress. 

E-Workbook 

Homepage 

0:52:14 

Observation 

Positive 

Structure 

Likes  that  it's  like  a  workbook  -  can  work  on  it  a  little  bit,  save, 

and  come  back  to  it  later. 

E-Workbook 

0:52:25 

Observation 

Structure  -  Layout 

Sees  that  the  pull-down  menu  is  on  the  right  and  she  can  pick 
which  topic  she  wants  to  look  at  in  more  detail. 

E-Workbook 

0:53:15 

Observation 

Thought  the  program  would  give  her  more  info/skills  but  then 
thinks  that  that's  probably  what  the  e-workbook  will  provide. 

E-Workbook 

0:54:23 

Observation 

Negative 

Content  -  Instructions 

Doesn't  understand  what  she's  supposed  to  do.  She  said  she 
would  have  to  read  it  again.  (53:44-56:20) 

E-Workbook:  50 

Benefits  of 

Managing  Your 

Stress. 
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P5-SM-DP 


R1P3-SM-DP-060211 


2/27 


Time 

Type 

Pos/Neg 

User  Event 

User  Event  Description 

Screen  Description 

0:02:02 

Other 

Structure  - 

Layout/Text 

Initially  appears  like  participant  will  go  to  Returning  User  but  catches  that 
he  needs  to  register  as  a  New  User 

Log-In 

0:02:13 

Participant 

prompted 

Prompted  participant  to  register 

Log-in 

0:09:44 

Other 

Note:  Would  have  screened  "out"  of  secondary  screening  questions 

Screening  Questions 

0:14:37 

Observation 

Positive 

Content  - 

Language 

Apologized  he  wasn't  commenting  on  the  program.  Said  the  questions  were 
easy  to  understand  and  answer  so  he  forgot  to  comment  as  he  went  along. 

Perceived  Stress 

0:16:32 

User  needs 
help 

Content  - 

Language 

Asked  participant  what  a  question  (QOL-  Having  and  Rearing  Children) 
meant.  Asked  him  what  he  thought  it  meant.  Said  that  he  just  didn't  read 
it  carefully  enough  &  now  he  understands  it.  Asked  him  to  explain  why  he 
selected  his  response.  His  justification  seemed  like  he  understood  the 
question  to  me. 

Quality  of  Life 

0:17:57 

Error 

Structute  - 

Layout 

Clicked  on  "next"  button  before  completing  questionnaire  items  on  second 
screen  of  questionnaire.  (Seemed  like  he  thought  he  already  answered 
the  items  since  screen  looked  the  same  as  the  previous  one).  Got  error 
message,  read  it,  and  completed  the  items  without  assistance. 

Quality  of  Life 

0:19:42 

Error 

Structure  - 

Layout/Text 

Participant  didn't  catch  that  the  previous  screen  asked  if  he  currently 
smoked  (which  was  "never"  for  him)  so  when  he  got  to  this  screen  (how 
long  ago  he  quit),  he  didn't  know  how  to  respond.  Suggested  participant 
return  to  the  previous  screen  and  he  noticed  his  error. 

Current  Smoking 

Habits 

0:21:02 

Observation 

Structure  - 

Text 

Suggested  that  this  definition  screen  of  exercise  activity  levels  could  be 
shortened/simplified. 

Exercise 

0:21:57 

Observation 

Structure  - 

Text 

Suggested  that  this  definition  screen  of  healthy  eating  could  be  more 
detailed. 

Healthy  Eating 

0:22:19 

Observation 

Content  - 

Language 

Thought  "sometimes"  should  be  an  additional  response  option  on  this 

screen. 

Healthy  Eating  -  eating 
number  of  calories  to 

reach  and  maintain 
healthy  weight 
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P5-SM-DP 


R1P3-SM-DP-060211 


2/27 


Time 

Type 

Pos/Neg 

User  Event 

User  Event  Description 

Screen  Description 

0:23:30 

Observation 

Content  - 

Language 

Commented  that  this  definition  screen  should  be  combined  with  the 
question  screen  (as  well  as  other  similar  definition/question  screens)  so  you 
don't  have  to  remember  the  definition  or  click  "previous"  button. 

Responsible  Drinking 

0:27:59 

Participant 

prompted 

Positive 

Content  - 

Language 

Asked  participant  if  the  program  made  sense  so  far.  Said  it  did. 

Program  Home  Screen 

0:31:32 

User  needs 
help 

Structure  - 

Layout/Text 

Said  he  was  "lost"  when  he  gets  to  the  program  welcome  screen. 

Reported  that  he  didn't  read  the  screen  thoroughly  because  it  seemed  like 
the  information  on  the  screen  was  presented  earlier  so  thought  he  could 
skip/skim.  Said  he  wasn't  sure  what  to  do  next  and  initially,  thought  he 
was  done  and  wouldn't  need  to  return  for  a  month.  Then  he  read  the 
screen  and  realized  he  needed  to  continue  and  selected  the  DP  program. 

Program  Home  Screen 

0:39:50 

Observation 

Negative 

Structure  - 

Text 

Said  that  some  screens  are  a  little  wordy.  Said  that  the  information  is  good 
but  might  not  be  good  keeping  someone's  attention. 

Advantages  and 
Disadvantages 

0:44:05 

Observation 

Content  - 

Language 

Said  for  clarity,  he  would  prefer  "sometimes"  instead  of  "occasionally"  as  a 
response  option. 

Your  Own  Experiences 

0:45:12 

Other 

System 

Went  "back"  2  screens  and  the  system  cleared  his  answers  to  the  first 
screen.  Confused  because  he  thought  he  was  on  a  new  screen  but  the 
questions  were  repetitive. 

Your  Own  Experiences 

0:46:43 

Observation 

Content  - 

Language 

Said  that  one  of  the  questionnaire  items  were  similar  to  another.  He  said  it 
made  him  concerned  about  how  he  answered  a  previous  item  because  he 
wanted  to  be  consistent. 

Your  Own  Experiences 

0:53:33 

Participant 

prompted 

Both 

Content  - 
Language/Te 

xt 

Asked  participant  what  he  thought  of  the  feedback.  Said  he  thought  it  was 
kind  of  basic  (e.g.,  go  for  a  walk)  especially  for  people  who  have  had 
previous  counseling.  But  he  likes  the  fill-in-the-blank  options  because  they 
helped  him  think  about  options. 

Your  Own  Experiences 

0:57:35 

Observation 

Positive 

System 

Commented  that  being  able  to  print  out  a  copy  of  his  report  was  good 
because  he  could  see  how  he's  doing  and  what  he  could  work  on. 

Your  Summary 

1:00:26 

Observation 

Structure  - 

Layout 

Commented  that  the  DP  and  SM  programs  look  similar. 

Stress  Management 

Activities 
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P5-SM-DP 


R1P3-SM-DP-060211 


2/27 


Time 

Type 

Pos/Neg 

User  Event 

User  Event  Description 

Screen  Description 

1:00:44 

Observation 

Structure/Co 

ntent 

Commented  that  real  program  users  wouldn't  have  to  do  the  two  programs 
back-to-back  (implying  it  was  a  lot  to  do  at  once).  Told  it  would  be  up  to  the 
user  but  that  they  could  return  or  do  them  at  once  depending  on 
preference. 

Stress  Management 

Activities 

1:24:59 

Participant 

prompted 

Both 

General 

Asked  what  he  thought  of  the  programs  in  general.  Said  he  thought  they 
were  good,  especially  the  fill-in  items  because  they  make  you  think.  When 
prompted,  he  said  there  was  nothing  he  saw  that  wasn't  appropriate  for 
Veterans.  When  prompted  if  program  would  be  helpful,  said  that  for  him,  it 
was  a  lot  of  text  on  each  screen. 

Program  Home  Screen 

BOLDED  user  event  descriptions  indicate  times  participant  may  have  had 
difficulty  with  program  because  he  didn't  read  the  text  thoroughly.  Also 
perhaps  an  issue  with  how  the  text  is  presented. 
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P6-DP 


R1P2-DP-060211 


2/27 


Time 

Type 

Pos/Neg 

User  Event 

User  Event  Description 

Screen 

Description 

0:02:06 

Other 

Note:  Initially  looked  like  participant  was  going  to  try  to  log-in  as  a 
Returning  User.  But  before  he  entered  anything,  he  saw  that  he 
needed  to  register  first  and  he  did  without  any  difficulty. 

Log-In 

0:05:17 

User  needs  help 

Content  - 

Instructions 

Confused  about  screening  criteria  (inpatient  treatment);  would've 
screened  out 

Screening 

Questions 

0:10:07 

Participant 

prompted 

Positive 

Structure  - 
Color/Layout; 
Content  - 

Language 

Asked  participant  if  he  understood  everything  because  he  hadn't 
been  making  any  comments.  Said  he  understands  the  questions. 
Commented  that  he  likes  the  colors  because  they're  soothing.  Also 
said  it  was  easy  to  navigate  the  program. 

General  Questions 

0:23:05 

Observation 

Negative 

Content  -  Specific 

Item 

Didn't  connect  with  an  item  (i.e.,  if  he  was  less  depressed  others 
would  expect  more  from  him).  Said  he  felt  the  opposite  way  (i.e., 
excells  when  others  expect  more  -  drill  sargeant,  coach). 

Advantages  & 
Disadvantages 
Questionnaire 

0:27:34 

Observation 

Negative 

Content  - 

Questionnaire 

Items 

Said  that  being  in  the  military,  being  male,  and  raised  in  the  culture 
he  was  raised  in,  he  wasn't  encouraged  to  express  things  so  some 
items  didn't  resonate  with  him. 

Your  Own 

Experiences 

0:30:38 

Other 

Negative 

Technical 

Difficulties 

Program  didn't  respond  if  the  "Next"  button  was  clicked  more  than 
once.  Had  to  close  out  screen  and  go  back  to  Home  Screen. 
Previously  happened  when  Pro-Change  was  not  informed  we  would 
be  doing  usability  testing.  However,  they  had  been  informed  of 
testing  this  time. 

Your  Own 

Experiences 

0:40:28 

Observation 

Positive 

Structure  -  Image 

Liked  picture  of  a  man  relaxing  in  a  hammock. 

Your  Summary 

0:44:01 

Observation 

Positive 

Content  - 

Informative 

Said  learning  about  how  to  use  disputing  statements  would  be  a 
good  idea 

E-workbook  -  Use 

the  ABCDs 
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P6-DP 


R1P2-DP-060211 


2/27 


Time 

Type 

Pos/Neg 

User  Event 

User  Event  Description 

Screen 

Description 

0:46:04 

Participant 

prompted 

Both 

Structure  -  Layout 

Asked  participant  if  he  would've  felt  he  needed  to/filled  in  all  7  text 
box  examples  if  he  was  at  home.  Said  he  might  not  have  even  gone 
to  the  e-workbook  at  all.  But  he  enjoys  "mental  stimulation"  so  it 
would  depend  on  if  he  had  the  time.  Said  some  links  sounded 
interesting. 

E-Workbook  - 

Increase  Positive 

Thinking 

0:49:11 

Observation 

Positive 

Content  -  Links 

Said  he  would  click  on  certain  e-workbook  links  out  of  curiousity. 

E-Workbook  - 
Thought  Stopping 

0:50:07 

Participant 

prompted 

Both 

Structure  - 
Color/Layout/Text 

Asked  participant  what  he  thought  of  the  program  in  general.  Said 
he  liked  the  colors.  Suggested  adding  drop-down  menus  at  the  top 
of  the  screens  so  options  are  easier  to  see.  Also  suggested  not 
putting  too  much  information  on  each  screen  since  that  can  be  off- 
putting  &  encourages  skimming.  Suggested  only  putting  summaries 
of  the  important  information  and  have  links/drop-downs  as  an 
option  for  those  that  want  more  detailed  information. 

(Program  Home 
Screen) 

0:52:03 

Participant 

prompted 

Positive 

Content 

Asked  participant  if  he  would  use  this  program.  Said  he  would 
because  he  is  trying  to  better  himself  after  experiencing  post¬ 
deployment  issues. 

(Program  Home 
Screen) 

0:52:49 

Other 

Confidentiality 

NOTE:  Participant  says  his  name 

(Program  Home 
Screen) 
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Time 

Type 

Pos/Neg 

User  Event 

User  Event  Description 

Screen  Description 

0:01:56 

Other  -  interviewer 

note 

Structure  -  layout 

No  difficutly  (other  than  Internet)  understanding  that  he  was  "not  a 
Returning  User"  and  needed  to  register  first.  (Side-by-Side  New 
User/Returning  User) 

Log-In 

0:05:28 

Observation 

Structure  -  text 

Initially  thought  he  was  supposed  to  click  on  which  of  the  trauma 
criteria  applied  to  him  but  then  he  noticed  the  "Yes/No"  options 
and  radio  buttons  at  the  bottom  of  the  screen. 

Screening  Questions 
(Trauma) 

0:11:32 

Other  -  interviewer 

note 

Structure  -  font  size 

Text  may  be  easier  to  read  if  the  font  size  is  increased;  however, 
this  may  affect  the  alignment  of  the  text.  For  example,  the 
questionnaire  items  may  take  up  too  much  room  on  the  screen  so 
the  participant  had  to  keep  scrolling  up  to  see  the  response 

e.g..  Military  Experiences 

0:14:52 

Observation 

Content  -  language 
(response  options) 

Had  difficulty  deciding  between  "never"  and  "several  days" 
response  options.  Said  there  should  be  a  middle  option  such  as  "1- 
2  days". 

General  Questions 

0:15:21 

Observation 

Structure  -  font  size 

Said  it  kind  of  bothered  him  that  the  "p"  in  the  word  "problems" 
was  cut-off.  (Likely  because  the  font  size  was  increased.) 

General  Questions 

0:16:28 

Observation 

Content  -  language 
(response  options) 

Said  the  questionnaire  items  sounded  like  they  should  have 
"Yes/No"  response  options  rather  than  "not  at  all"  etc.  (Even 
though  the  question  asked  "How  often...".)  Agreed  that  response 
options  could  be  more  specific  like  "1-2  days  a  week",  etc. 

General  Questions 

0:19:49 

User  needs  help 

Content  -  language 

Assumed  the  questions  had  to  do  with  combat  since  the  heading 
was  "Combat  Experience".  Wasn't  sure  how  to  answer  items  if  he 
wasn't  in  combat.  Assisted  participant  to  think  of  items  as 
pertaining  to  his  military  service  (as  the  instructions  indicate)  rather 
than  during  combat.  Participant  suggested  changing  the  heading  to 
"Combat  or  Military  Experience." 

Combat  Experience 

0:23:23 

Observation 

Structure  -  font  size 

(Related  to  increased  font  size)  said  that  response  options  could  be 
repeated  on  the  side,  etc.  so  participants  don't  have  to  keep 
scrolling  up.  (Didn't  notice  that  response  options  appeared  if 
cursor  was  over  the  radio  button.) 

Quality  of  Life 
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Time 

Type 

Pos/Neg 

User  Event 

User  Event  Description 

Screen  Description 

0:31:40 

Participant  Prompted 

Content  -  language 

Asked  participant  if  he  understood  what  he  needed  to  do  if  he  was 
in  the  feasibility  study.  Said  he  thought  participants  should  log-in 
MORE  than  once  a  month  since  their  status  may  change.  Thought 
that  the  Thank  You  screen  seemed  like  a  class  syllabus  and  that  it 
might  deter  participants  because  it  looks  like  a  lot  of  work.  Said  he 
thought  the  word  "compensation"  sounded  like  he  was  goingt  o  be 
"sold"  something  and  it  put  his  guard  up. 

Thank  You 

0:35:12 

Other  -  interviewer 

note 

Positive 

Content  -  language; 
Structure  -  layout 

Understood  he  was  supposed  to  click  on  a  program  (i.e.,  Stress 
Management)  and  did  not  have  to  be  prompted  to  continue. 

Program  Homepage 

0:37:21 

Observation 

Positive 

Content  -  language 

Said  he  liked  that  the  program  said  "the  stress  management 
program  can  help"  on  the  first  screen.  Siad  it  was  "a  good  pitch." 

About  This  Program 
(SM) 

0:37:41 

Observation 

Content  -  language  in 
links;  Structure  -  text 

Clicked  on  the  link  to  the  TTM  to  find  out  the  definition.  Said  the 
pop-up  box  had  more  information  that  what  he  was  looking  for  and 
initially  said  it  didn't  provide  the  definition.  Later,  he  said  that  he 
had  heard  of  the  stages  before  and  called  it  "PCPM"  (as  an 
acronym)  and  was  able  to  describe  the  stages. 

About  This  Program 

0:39:45 

Observation 

Positive 

Structure  -  image 

Said  he  liked  the  image  of  his  stage  of  change. 

Your  Stage  of  Change 

0:41:35 

Error 

Structure  -  error 

message 

Missed  a  questionnaire  item  and  received  an  error  message.  Saw 
which  item  he  missed  (thought  he  answered  it  but  realized  he 
accidentally  answered  the  wrong  item)  and  answered  it  without 
assistance/prompting. 

Stress  Management 

Activites 

0:43:56 

Other  -  interviewer 

note 

Content  -  language 

Prior  to  usability  testing,  participant  completed  a  hard  copy  stress 
management  questionnaire  and  didn't  know  what  the  term  "poke 
fun"  meant.  He  noticed  that  the  same  item  appeared  at  this  point 
in  the  program.  Had  he  not  be  explained  the  meaning  of  the  term 
earlier,  he  might  not  have  known  how  to  answer  it  or  would  have 
guessed. 

Stress  Management 

Activites 

0:45:32 

Observation 

Negative 

Content  -  language 

Said  it  wasn't  clear  that  the  feedback  screens  were  giving  him 
suggestions  based  on  his  previous  responses.  Said  the  headings 
(e.g.,  Activities  &  Strategies  -  Being  Prepared  and  Planning  Ahead) 
didn't  described  that  it  was  tailored  feedback. 

Activities  &  Strategies 

0:50:09 

User  needs  help 

Content  -  instructions 

Initially  started  answering  the  questionnaire  items  without  reading 
the  instruction  question  at  the  top  of  the  screen  &  therefore  said 
the  items  didn't  make  sense.  Suggested  participant  read  the 
instruction  question  and  then  he  said  he  understood  the  items. 

Pros  &  Cons 
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Time 

Type 

Pos/Neg 

User  Event 

User  Event  Description 

Screen  Description 

0:50:35 

Observation 

Structure  -  font  size 

(Font  size  finally  changed  to  small.)  Said  he  could  read  the  smaller 
font  because  he  had  his  glasses  on  and  liked  that  he  could  see  all  of 
the  items  on  the  screen  without  having  to  scroll  down. 

Pros  &  Cons 

0:52:40 

Observation 

Content  -  language 
(response  options) 

(Again)  said  the  wording  is  unclear  on  the  questionnaire  items 
(importance  of  the  items  to  effectively  manage  stress).  Said  he 
thought  they  sounded  more  like  "Yes/No"  items. 

Pros  &  Cons 

1:03:17 

Observation 

Structure  -  image 

Image  -  said  the  picture  (man  with  zen  garden)  was  distracting 
because  he  couldn't  tell  what  it  was.  He  said  it  kept  drawing  his 
attention  away  from  the  text  because  he  wanted  to  know  what  the 
man  was  doing. 

Your  Strategies  for 
Change 

1:05:52 

Observation 

Negative 

Content  -  language 
(unclear) 

Thought  that  when  the  previous  screen  said  that  it's  time  to  "pick  a 
start  date",  it  meant  that  this  screen  would  give  him  a  date  or  have 
him  select  a  start  date,  etc.  He  was  confused  and  disappointed  that 
he  didn't  see  what  he  expected.  He  said  actually  picking  a  start 
date  within  the  system  would  increase  his  commitment  to  change. 

Thank  You 

1:08:33 

Participant  Prompted 

Asked  if  he  would  do  both  programs  back-to-back  or  return  later  if 
he  were  in  the  feasibility  study  doing  this  at  home.  He  said  he 
would  most  likely  do  the  programs  back-to-back. 

Program  Homepage 
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Time 

Type 

Pos/Neg 

User  Event 

User  Event  Description 

Screen 

Description 

0:05:58 

Other  - 

interviewer 

note 

Computer 

literacy 

Realized  that  participant  was  not  screened  and  he  was  not  comfortable  using 
the  computer/Internet  (would  have  been  ineligible  for  usability  interview). 
Decided  to  let  him  continue  and  test  the  system  as  a  non-computer  literate 
user  with  our  technical  assistance.  Had  difficulty  entering  information  and 
needed  a  lot  of  assistance. 

Log-in 

0:14:16 

Error 

Content  - 

language 

Answered  "yes"  to  screening  question  and  would  have  incorrectly  screened 
out  (of  feasibility  study).  Said  it  was  because  he  has  "depression"  and  didn't 
understand  what  "manic-depression"  was.  (Perhaps  there  could  be  a  link  to 
a  definition  of  less  familiar  terms  like  these.) 

Screening 

Questions 

0:18:49 

User  needs 
help 

Structure  - 
layout 

Tried  to  click  on  a  bullet  to  respond  on  an  instruction  screen  (no  questions). 

Healthy  Eating 

0:29:16 

User  needs 
help 

Content  - 
language 

Confused  about  the  staging  screen.  Thought  it  was  asking  how  long  he's 
been  depressed  versus  when  he  was  planning  on  starting  depression 
prevention  strategies. 

About  this 

Program  (DP) 

0:29:47 

Observation 

Structure  - 

layout 

Again,  tried  to  click  on  a  bullet  to  respond  on  an  instruction  screen  (no 
questions). 

About  this 

Program 

0:37:16 

User  needs 
help 

Structure  - 
layout 

Again,  thinks  he  needs  to  click  on  the  bulleted  items  to  respond  even  though 
the  screen  was  only  informational. 

Depression 

Prevention 

Activities 

0:38:51 

User  needs 
help 

Structure  - 

layout/text; 
Content  - 

questionnaire 

items 

Asked  for  clarification  regarding  the  questionnaire  items.  Suggested  he  read 
the  main  question/instructions.  He  did,  and  was  then  able  to  answer  the 
items. 

Advantages  & 
Disadvantages 

0:40:03 

Participant 

prompted 

Structure  - 
progress  bar 

Laurel  asked  if  he  understood  what  the  bar  was  at  the  bottom  of  the  screen. 
Participant  guessed  it  gave  a  message  of  how  depressed  he  was.  Laurel  told 
him  it  was  a  progress  bar  and  then  he  understood. 

Advantages  & 
Disadvantages 
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Time 

Type 

Pos/Neg 

User  Event 

User  Event  Description 

Screen 

Description 

0:54:04 

User  needs 
help 

Content  - 

language 

He  wasn't  sure  what  he  needed  to  do  on  this  screen.  Thought  the  "report" 
button  might  be  asking  him  to  report  "on"  something.  Was  told  to  click  on  it 
to  get  his  personalized  report. 

Thank  You 

0:55:05 

Other - 

interviewer 

note 

Technical  issue 

Wasn't  able  to  enlarge  screen  so  had  to  adjust  to  100%  so  could  see  all  the 
text  in  the  screen. 

Personal  Health 
Report 

0:58:24 

Participant 

prompted 

Both 

Asked  participant  if  he  thought  this  program  would  be  helpful  to  him. 
Responded  that  he  wasn't  sure  if  he  would  turn  to  this  program  because  of 
his  particular  situation,  but  he  thought  it  did  have  some  helpful  suggestions 
which  might  be  more  helpful  to  others.  Thought  information  could  be  more 
specific  and  include  other  areas  of  depression  (e.g.,  hopelessness,  living 
situation). 

Personal  Health 

Report 
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Time 

Type 

Pos/Neg 

User  Event 

User  Event  Description 

Screen 

0:03:07 

Other  - 

interviewer  note 

Structure  - 

layout 

Said  she  needed  a  Log-In  Name  but  then  went  to  More  Information  about 
the  program. 

Log-In 

0:04:32 

Other - 

interviewer  note 

Structure  - 

layout 

After  participant  closed  the  Study  Fact  Sheet/More  Information,  she  said 
she  would  register  as  a  New  User.  Prompted  her  that  she  could  enter  any 
information  here. 

Log-In 

0:05:33 

User  needs  help 

Structure  - 

layout 

Selected  the  three  Security  Questions  from  the  drop-down  boxes  but  did 
not  provide  any  answers.  Received  an  Error  Message  and  didn't  know  how 
to  resolve  the  error.  Was  prompted  that  she  didn't  answer  the  security 
questions  and  needed  to  register  again. 

Log-In 

0:08:23 

Other - 

interviewer  note 

Would  have  screened  out  at  Trauma  Criteria. 

Screening 

Questions 

0:10:47 

Participant 

prompted 

Structure  - 
layout/imag 

es 

Asked  participant  what  she  thought  of  the  "new"  homepage  (revised 
recently  by  Pro-Change).  Said  she  would  have  suggested  adding  pictures, 
but  then  she  said  she  scrolled  down  and  saw  the  pictures  at  the  bottom  of 
the  screen.  Said  she  would  make  the  pictures  more  prominent  (e.g.,  on  the 
sides  or  top  of  the  screen)  and  make  them  more  relevant  to  the  programs 
(e.g.,  someone  putting  out  a  cigarette). 

New  Homepage 

0:14:15 

Participant 

prompted 

Content  - 

language 

Asked  participant  to  verbalize  what  the  homepage  was  instructing  her  to 
do.  Said  she  thought  it  should  say,  "Now  that  you've  completed  your 
assessment/looked  at  your  personalized  report,  the  next  step  is..."  so  the 
participant  knows  exactly  what  to  do  next.  Said  she  would  also  add  go  to 
programs  "to  the  left  of  the  screen"  so  the  participant  knows  where  they 
need  to  go  to  select  their  programs. 

New  Homepage 

0:17:19 

Participant 

prompted 

Content  - 

language 

Asked  participant  what  she  thought  of  the  "old"  homepage.  Said  she 
thought  it  explained  what  to  do  more  thoroughly  and  was  easier  to 
understand  than  the  revised/new  homepage. 

Old  Homepage 
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Time 

Type 

Pos/Neg 

User  Event 

User  Event  Description 

Screen 

0:18:04 

Participant 

prompted 

Structure  - 

image 

Asked  participant  what  she  thought  of  the  image  and  she  provided  several 
suggestions.  She  said  the  colors  could  be  more  vibrant,  could  include 
different  images,  and  the  images  could  be  moved  to  the  top  or  sides  of  the 
screen.  She  said  in  general  this  screen  is  very  "sterile"  and  uninteresting. 

Old  (and  New) 
Homepage 

0:20:11 

Observation 

Structure  - 

image 

Said  she  would  add  an  image  of  a  really  stressed  out  person  (like  hair's 
being  electrocuted)  on  one  side  of  the  screen  and  a  really  relaxed  person 
on  the  other  side  -  like  a  before  and  after  program. 

Stress 

Management 

Homepage 

0:21:09 

Observation 

Structure  - 

image 

Would  use  pictures  along  with  the  text. 

About  This 

Program 

0:22:26 

Observation 

Content  - 

language 

Initially  she  thought  the  Stages  of  Change  screen  was  about  the  different 
types  of  stress.  Clicked  on  the  TTM  link  to  find  out  more  information.  Said 
the  information  could  be  simplified  but  was  understandable. 

About  This 

Program 

0:24:35 

Observation 

Positive 

Content  - 

language 

Said  this  screen  was  explained  well. 

About  This 
Program  (3rd 
screen) 

0:28:41 

Participant 

prompted 

Content  - 

language 

Asked  participant  if  she  knew  what  the  term  "poke  fun"  meant.  She 
accurately  replied  that  it  meant  to  laugh  at  sources  of  stress  in  your  life. 

Stress 

Management 

Activities 

0:29:32 

Observation 

Content  - 

language 

Said  the  information  on  this  screen  seemed  repetitive  &  said  last  question 
could  be  reworded  to  "to  see  more  progress..."  instead  of  "to  make  even 
more  progress." 

Activities  and 

Strategies 

0:31:23 

Observation 

Content  - 

language 

Said  "Taking  a  Healthier  Approach"  screen  should  provide  some  positive 
examples.  Said  the  screen  could  say  "in  order  to  avoid  (this  category)"  and 
then  provide  some  options. 

Activities  and 

Strategies  - 
Taking  a  Healthier 
Approach 
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Time 

Type 

Pos/Neg 

User  Event 

User  Event  Description 

Screen 

0:33:41 

Observation 

Content  - 

language 

Thought  this  screen  focused  too  much  on  stress  and  the  negative  instead  of 
taking  your  mind  off  of  the  stress.  Said  should  have  positive  ideas  (e.g.  go 
for  a  walk,  take  a  bath,  drink  a  cup  of  tea,  etc.). 

Activities  and 

Strategies  - 
Focusing  on  Your 
Response  to 

Stressful  Events 

0:35:36 

Observation 

Content  - 

language 

Thought  some  questionnaire  items  could  use  skip  patterns  since  she 
thought  if  she  agreed  to  one  item,  it  would  make  others  irrelevant. 

Pros  &  Cons 

0:38:17 

Observation 

Content 

Thought  the  program  was  appeal  to  more  people  if  it  was  tailored  to 
participants  in  other  areas  (e.g.,  weight  management). 

Weigh  the  Pros  & 

Cons 

0:40:31 

Observation 

Positive 

Content  - 

language 

Thought  the  insturctions  and  questionnaire  screens  made  sense. 

How  Confident 

Are  You 

0:46:38 

Observation 

Content  - 

language 

Thought  the  questionnaire  items  were  "sterile"  and  didn't  reflect  what  she 
considered  stressful  experiences. 

Your  Own 

Experiences 
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